Editing large text files 2

[FlashGorden]

Hi

I have a text file with a thouand lines each line represents a record. I need to create some test data based on this file to upload to an Oracle db. I need to create 5000 of these files and use a counter to change the primary key field to a unique value.

The fields are pipe delimited (|) and the key is the 3rd field. any help would be greatly appreciated.

Thanks

 

[bjverzal]

How many total fields are there ? This will not be too hard to do.

Bill.

 

[FlashGorden]

There around 40 fields

 

[bjverzal]

Is the 3rd field numeric ?

If so, do the following:

<some file> is the name of your file

#!/bin/ksh
file=1
last_line=`tail -1 <some file>`
pkey=`echo $last_line|cut -f3 -d&quot;|&quot;`
ff2=`echo $last_line|cut -f4- -d&quot;|&quot;`
ff1=`echo $last_line|cut -f1-2 -d&quot;|&quot;`
line=0
maxline=`cat <some file>|wc -l|awk {'print $1'}`

while [ $file -le 5000 ] ; do
line=`expr $line + 1`
if [ &quot;$line&quot; -ge &quot;$maxline&quot; ] ; then
file=`expr $file + 1`
> some_file.$file
line=1
fi
pkey=`expr $pkey + 1`
lineout=&quot;$ff1|$pkey|$ff2&quot;
echo $lineout >> somefile.$file
done

 

[GOSCO]

I hope I understand correctly anyway I took a file like this:

field1|field2|field3
field1|field2|field3
field1|field2|field3

and turned it into this:

field1|field2|1|field3
field1|field2|2|field3
field1|field2|3|field3

The code:

awk -F&quot;|&quot; ' BEGIN { pkey=0 }
{
pkey++;
printf(&quot;%s|%s|%d|%s\n&quot;,$1,$2,pkey,$3);
}' < log > /tmp/out

This program gives the output as above. If you need an explanation, let me know!