Finding similarities 2

[Mag0007]

I have a list similar to this:

one
two
three
one
four
three
two
five
three
four
two

I need to just output non duplicates like this (order does not matter):

one
two
three
four
five


Is there a way to do this? or should I just store this list in a database and run a SELECT with distinct?

 

[Mike042]

Have you looked at the man pages for sort and uniq ?

I hope that helps.

Mike

 

[Mag0007]

Mike:

Nice! uniq looks like it will do the trick!!! You rock!

thanks!

 

[p5wizard]

well...

uniq gives unique consecutive lines

your list:

one
two
three
one
four
three
two
five
three
four
two

run thru uniq doesn't change a whole lot...

you have to

sort|uniq

or

sort -u


HTH,

p5wizard

 

[Mag0007]

thx p5wizard!

first I must get everything in a list

file looks like this now:

one two three one two three four one one two
two two one two three four five
five two one two two three three two three four five

 

[PHV]

A starting point:
awk '{for(i=1;i<=NF;++i)++a[$i]}END{for(i in a)print i}' /path/to/input > output

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[p5wizard]

starting to look a lot like homework questions...


HTH,

p5wizard

 

[Mag0007]

p5wizard:

I wish it was a homework problem

PHV:
VERY NICE! with some minor adjustments, this should work!