Grep: Return a user message when no hits 1

[kristo5747]

I have ASCII files to parse that 48 hours old or more ; I can identify them like so

    find . -name "FILE*TXT"  -mtime +1 -exec ls -ltas '{}' ';'

Some files have a list of hardware errors (we test electronic components), some have none. If the file name has no errors, I still want to display a message like so

        grep ^err R*VER && echo "No error"
    FILEA.TXT:err	->USB	3910	err	
    FILED.TXT:err No Error

This grep statement works but it seemingly overrides the find() statement above if I run both at the same time... How can I combine the two statements to create a report that lists the filename and error(s) like so

    FILEA.TXT Button	3320	err
    FILEB.TXT USB	3235	err
    FILEC.TXT IR Remote	2436	err
    FILED.TXT No error

Is it possible to return "No error" with the file name without error? Thanks in advance for your help.

 

[LKBrwnDBA]

Are you sure your "find" statement works ok?

Anyway, try this:

for f in `find . -name "FILE*TXT"  -mtime +1 -exec ls -ltas {} ';'`
do
 grep ^err $f 
 [ $? -ne 0 ] && echo "$f No error"
done

----------------------------------------------------------------------------
The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb

 

[Annihilannic]

Or:

for f in `find . -name "FILE*TXT"  -mtime +1 -exec ls -ltas {} ';'`
do
 grep ^err $f || echo "$f No error"
done

Annihilannic.

 

[kristo5747]

A friend, who's much smarter than me, came up with this

find . -name "FILE*TXT" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;

It works like a charm!

Thanks for taking the time.

 

[kristo5747]

After running the command

find . -name "FILE*TXT" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;

my files look like this

RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922 	
...

Using the tilde (~) symbol as delimiter, can I extract the fields I want so I get an output like this

RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error RRR1 COS P
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922 RRR1 COS F 	
...

I tried the following

find . -name "R*VER" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;|awk -F~ '{print $0}{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}'

but it does not work; Instead, it produces this output (not all fields shown here...)

RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER RRR1 COS P
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER RRR1 COS F

I'd like to do this is one pass so I can generate a SQL script with INSERT statements...Can it be done?

 

[Annihilannic]

Try using awk -F '~' ... because ~ has special meaning to the shell (will be replaced with your home directory). Otherwise it looks like it should work.

Annihilannic.

 

[kristo5747]

No difference. Statement works but in two passes instead of once....

 

[Annihilannic]

If you only want to output 3 fields, e.g. "RRR1 COS P", why are you printing so many of them in your awk script?

When I run your code I get this output:

$ ls -l
total 8
-rw-r--r-- 1 srochfor users 20 Jun 15 09:39 RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER
-rw-r--r-- 1 srochfor users 31 Jun 15 09:39 RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER
$ cat *1.VER
Nothing to see here
$ cat *9.VER
This one didn't work
err 3922
$ find . -name "R*VER" -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;|awk -F'~' '{print $0}{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}'
./RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER:err 3922
./RRR1  COS     F       R16-500 000907009757    20110607
./RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER:No error
./RRR1  COS     P       R22-200 029053662549    20110607
$

Same results as you?

Annihilannic.

 

[kristo5747]

YES!! Same results as me.

I am pulling whatever hair I have left to combine, for instance,

./RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER:err 3922
./RRR1  COS     F       R16-500 000907009757    20110607

one one single line so it looks like this

RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER:err 3922 RRR1 COS F R16-500 000907009757    20110607

 

[kristo5747]

If you only want to output 3 fields, e.g. "RRR1 COS P", why are you printing so many of them in your awk script?

Actually, I need to display eight fields.

I tried to keep my post as concise as possible so I used only three fields here.

 

[Annihilannic]

The only differences I can see between those two lines are the ./ prefixes and some spaces instead of tabs... is that right?

The former can be fixed with a sed replacement or by using something like sub(/^.\//,"",$1) in your awk script.

The latter can be fixed by removing the \t tabs and just using spaces.

Ahhh!! Eureka moment... my web browser is wrapping your second example so it also looks like it is on 2 lines at my end. In that case, just combine both print statements into one:

 '{sub(/^.\//,"",$1);print $0,$1,$2,$7,$8,$9,$10}'

I have used commas here which means that awk just uses the default output separator... a space.

Annihilannic.

 

[Annihilannic]

Actually, slight change:

'{t=$0;sub(/^.\//,"",$1);print t,$1,$2,$7,$8,$9,$10}'

This is to cater for the fact that using a sub() on a field number causes the value of $0 to get mangled, so I save it to a variable first for use later.

Annihilannic.

 

[Annihilannic]

Geez, I'll get there in the end. This is starting to look pretty clunky... I do like awk but it can be very awkward sometimes!

This version will remove *both* occurences of "./".

'{t=$0;sub(/^.\//,"",t);sub(/^.\//,"",$1);print t,$1,$2,$7,$8,$9,$10}'

Annihilannic.

 

[kristo5747]

The slash prefixes and spaces are not a problem: I can fix that later. Either way, it seems your solution would work. All I have to do is replace my awk statement with yours...I'll check tomorrow and update this post. Thx for your time.

 

[kristo5747]

Sorry but I was unable to make your suggestion to work

I came up with this (with help)

for myfile in `find . -name "R*VER" -mtime +1`
do
     somestr=`grep -H ^err $myfile || echo -e "$myfile\tNo error"`
     echo "$somestr" |sed 's:./::;s:tmp/::;s/:/\t/;'
     echo $somestr | awk -F~ '{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}' | sed 's:./::;s:tmp/::;'
done

It is more concise but still outputs on two lines i.e.

RRR1~COS~COSMETICS~ETT04~jgmdtv175~2~P~R15-300~000876523911~20110607~102159.VER	No error
RRR1	COS	P	R15-300	000876523911	20110607
RRR1~COS~COSMETICS~99537~jgmdtv132~2~P~R22-200~029051946829~20110607~101331.VER	No error
RRR1	COS	P	R22-200	029051946829	20110607

I am looking to generate dynamically a SQL script with insert statements like so:

INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8)
VALUES ('RRR1~COS~COSMETICS~ETT04~jgmdtv175~2~P~R15-300~000876523911~20110607~102159.VER','No error','RRR1','COS','P','R15-300','000876523911','20110607')

All I would need to do is wrap both lines but I am not clear how to do it (still learning awk/sed).


Any input welcome. Thanks.

 

[kristo5747]

I was able to make your suggestion work like so

find . -name "R*VER" -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;|awk -F~ '{t=$0;sub(/^.\//,"",t);sub(/^.\//,"",$1);print t,$1,$2,$7,$8,$9,$10}'

but it is strangely working only for the "No error found" scenario

tmp/RRR1~COS~COSMETICS~99537~jgmdtv132~3~P~R22-200~029051891686~20110607~094714.VER:No error tmp/RRR1 COS P R22-200 029051891686 20110607

When errors are found, this is output

 tmp/RRR1 COS F R16-300 000894980523 201106076-300~000894980523~20110607~084053.VER:err	->IR Remote Key 1	3310	err

Any idea?

PS: much appreciative of your time

 

[Annihilannic]

The extra line of output is coming from the middle line in the loop; you need to assign this output to a variable if you want to store rather than print it, e.g.

for myfile in `find . -name "R*VER" -mtime +1`
do
     somestr=`grep -H ^err $myfile || echo -e "$myfile\tNo error"`
     somestr=`echo "$somestr" |sed 's:./::;s:tmp/::;s/:/\t/;'`
     echo "$somestr" | awk -F~ '{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}' | sed 's:./::;s:tmp/::;'
done

You could do all of that in one pipeline as follows:

for myfile in `find . -name "R*VER" -mtime +1`
do
     ( grep -H ^err $myfile || echo -e "$myfile\tNo error" ) |
                sed 's:./::;s:tmp/::;s/:/\t/;' |
                awk -F~ '{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}'
done

Adding the final bits:

for myfile in `find . -name "R*VER" -mtime +1`
do
     ( grep -H ^err $myfile || echo -e "$myfile:No error" ) |
                sed 's:./::;s:tmp/::;' |
                awk -F: '
                        BEGIN { q="\047" }
                        {
                                file=$1
                                result=$2
                                split(file,a,/~/)
                                print "INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8) VALUES (" q file q "," q result q "," q a[1] q "," q a[2] q "," q a[7] q "," q a[8] q "," q a[9] q "," q a[10] q ")"
                        }
                '
done

Assigning the octal code for a single quote to 'q' saves having to either put lots of octal codes in the print statement, or do lots of fancy escaping to prevent the single quotes from being treated as the end of the awk script.

Annihilannic.

 

[Annihilannic]

Seems like we cross-posted. That solution adapted for the neater find without a for loop is:

find . -name "R*VER" -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \; |
        awk -F: '
                BEGIN { q="\047" }
                {
                        file=$1
                        result=$2
                        sub(/^.*\//,"",file)
                        split(file,a,/~/)
                        print "INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8) VALUES (" q file q "," q result q "," q a[1] q "," q a[2] q "," q a[7] q "," q a[8] q "," q a[9] q "," q a[10] q ")"
                }
        '

Annihilannic.

 

[kristo5747]

BRILLIANT!!!

Thanks so much. It works!!