Pure logic puzzle 1

[dickiebird]

Hi Guys
With a flat file consisting of :
0001,123
0002,023
0002,989
0003,123
0004,093
0005,030
0005,987
0006,030
0007,939 etc etc

I want to print all unique field 1, but only the second listed where there's a duplicate field 1.
So the above would end up like :
0001,123
0002,989
0003,123
0004,093
0005,987
0006,030
0007,939 etc etc
The first listed 0002 has gone , as has the first 0005.
I got this far and then cried :

awk ' BEGIN {FS = "," }
{
if(NR==1) # 1st line is OK to print anyway
{
print $0;
next;
}
if (f1sav=$1)
{
print $0;
}
else
{
print f0save
}
f1sav=$1;
f0save=$0;
}' allext > allexta

Whaddya think I need - apart from a brain implant ????
DB
Dickie Bird
db@dickiebird.freeserve.co.uk

 

[vgersh99]

If the oupur order does NOT matter...

BEGIN {
FS=","
}

{ arr[$1]=$0; }

END {
for (i in arr)
printf("%s\n", arr);
}

 

[dickiebird]

Hi vgersh99
Your method worked fine - I ran a sort on it to restore the order. How does the loading of an array work, though ?
DB
PS I also got it with this eventually
awk ' BEGIN {FS = "," }
{ if(NR==1)
print $0;
if (f1sav!=$1)
print f0save;
f0save=$0;
f1sav=$1;
}
END { print f0save;
}' allext > allexta

Dickie Bird
db@dickiebird.freeserve.co.uk

 

[vgersh99]

Hi,

awk uses the associative arrays. The array is loaded using the first colomn as it's INDEX and the entire record as its content. Any subsequent records with repetetive first colomns overwrite the the existing entry in the array [if one exists].

The output is produced by iterating through the array content.

HTH

vlad

 

[bigoldbulldog]

If this is your input (3 repeats and on term)

0001,123
0002,023
0002,989
0002,999
0003,123
0004,093
0005,030
0005,987
0006,030
0007,939
0007,939

and your output is

0001,123
0002,999
0003,123
0004,093
0005,987
0006,030
0007,939

then this sed script works

sed '
1{
${p;q;}
h
}
1!{
x
G
/^\([^,]*\),.*\n\1.*/{
s/.*\n//
$p
b
}
$!s/\n.*//
p
}
' allext > alltexta

Cheers,
ND

 

[bigoldbulldog]

Whops, that's
sed -n '
...

ND

 

[dickiebird]

Thanks guys - your help is very much appreciated
DB
;-)
Dickie Bird
db@dickiebird.freeserve.co.uk