help manipulating results from grep

[rico7474]

Hello all,
I'm looking for a way to manipulate the results from the grep command. I'm on a unix system, and I have a file (temp.txt) containing rows such as:

./misc/dir/file1.htm,123
./misc3/dir2/file1.htm,89
./misc/dir/file2.doc,3489
./file3.htm,349
./misc3/dir3/file5.doc,89
(etc)

I want something that will grab only the filename (eg. 'file1.htm') and write only the unique filenames to a new file (no duplicates). I can write a perl script to do this but I was wondering if there was a unix one-liner I could type on the command line to accomplish this.

That is, how do you trim the result from grep (in my perl script I use s/^.*\/(.*),.*/$1/ ), and would you then pipe this to the uniq command?

Thanks in advance,
Scott.

 

[tdatgod]

grep whatever | awk 'BEGIN {FS=/} {print $NF}' | sort -u

 

[rico7474]

Thanks tdatgod,

It didn't quite work like I needed but I made some modifications and it does seem to do the job. Here's what I have:

grep , filename | awk 'BEGIN{FS=","}{print $1}' | awk 'BEGIN{FS="/"}{print $NF}' | sort -u

I'm not sure if I needed to pipe to a second awk to split the string again but this seems to do the job.

Thanks for your help.

 

[cspilman]

rico7474,

This should work for you also:

cat filename | xargs -i basename {} | awk -F, '{print $1}' | uniq


Regards,
Chuck

 

[tdatgod]

Sorry didn't see the ','. Yes you need to get the first element some how before the sort. Wether you do it first or last doesn't matter as long as the second field doesn't have '/' in it.


I typically would use cut


grep what | cut -d',' -f1 | awk -F/ '{print $1}' | sort -u

or

grep what | awk -F/ '{print $1}' | cut -d',' -f1 | sort -u

both produce the same output assuming the second field doesn't ever contain a '/'.

 

[rico7474]

Thanks Chuck,
It took me a little while to get it working, just some minor errors on my part. Also, I could not get the uniq cmd to operate correctly (using option -u with it as well). It doesn't seem to want to filter the duplicate records. I'm just going to play with it for a while. Thanks for your help.

tdatgod - I substituted the $1 with $NF and it works great.

Cheers all.