Help me about "nawk"!!

[piggy213]

Sorry,my english is poor.
I describe my question clearly as soon as possible.
If I have a text file like this:
.............................
.............................
Fri Sep 27
ORA012:error 12345 in xxjob....
ORA014:normal 23455.....
Sat Sep 28
ORA012:normal 23456....
ORA014:error 34512 in xxjob...

Now,if I wanted to use C shell to find "yesterday error message" and print it, just like:
Fri Sep 27
ORA012:error 12345 in xxjob....
How should I use "nawk" to find out yesterday error message?
Thank you very much!

 

[sedj]

OK, sorry - I guess this response is as much about answering your query as it is answering my own....

(I'm a java programmer getting back to shell due to lack of java jobs and getting hired for old shell abilities which are rusty !!!)

apologies if the following is crap :
but from my days when I used to play with awk/sed/perk/shell etc ....

If in a korn shell for instance (maybe if the variable MYDATE was entered on command line),

THELINEIWANT = ""
for LINE in `cat myErrors.log`
do
awk -f ' if (LINE == "MYDATE&quot {
THELINEIWANT = "TRUE"
}
if (THELINEIWANT == "TRUE&quot {
THELINEIWANT = LINE
print LINE
print THELINEWANT
THELINEIWANT = "False"
}
else {
print "not the date I wanted"
}
'
done


Or is that complete crap and I should go back to java, resign my new job, and buy a small apartment in a padded cell......??

 

[Daedelus]

Sedj:
Don't know about the padded cell, but you definitely need to study the awk manpages for a while. I'm afraid the errors are numerous.

Piggy213:
I suggest the following script, which automatically finds yesterday's date as well. It works in Korn, and I believe it will also work in C shell, though I don't know for sure. I got the trick for finding yesterday's date from [faq80-953].

It assumes that the dates in your file are always formatted as you showed and are in order, and that single digit days are shown as "Fri Sep 7". If they are shown as "Fri Sep 07", change the "%e" in the date command to "%d". "myErrors.log" is your text file:

Today=`date +"%a %b %e"`   
TZ=`expr $TZ + 24`       #Move timezone up 24 hours
Yesterday=`date +"%a %b %e"` 

nawk '
   BEGIN {rightday=0}
   $0 == yesterday {rightday=1; print}
   /:error/ && rightday == 1 {print}
   $0 == today {exit}
   ' yesterday="$Yesterday" today="$Today" myErrors.log

that should give you any :error lines between yesterday's date line and today's date line. I used "expr" to add 24 to TZ, date's timezone parameter, because I am not sure how to add in C shell. In the Korn shell (ksh) you can use ((TZ += 24)) in its place. (In fact I suggest you just add

#! /bin/ksh

to the front of your script and forget about the C shell!)

 

[piggy213]

Hi Daedulus,
Thanks for your help.
I was forced to use C shell. So about the "yesterday" point I used the stupid way to calculate it.
This is the original file:
Mon Sep 30 02:02:23
ORA014:normal 23455.....
Mon Sep 30 09:31:21
ORA012:normal 23456....
ORA014:error 34512 in xxjob...
Mon Sep 30 12:31:21
ORA012:normal 23456....
ORA014:error 34512 in xxjob...
Mon Sep 30 22:02:23
ORA014:normal 23455.....

The following are my program which combine your suggestions.

#!/bin/csh -f
set month = `date | cut -f2 -d' '`
set day = `date | cut -f4 -d' '`
@ no = $day - 1
if ( "$day" == 1 && "$month" == Oct ) then
if ( "$month" == Jan || "$month" == Mar || "$month" == May || "$month" == Jul || "$month" == Aug || "$month" == Oct || "$month" == Dec ) then
set no = "30"
set month = "Sep"
...................
...................
else
set no = "31"
...................
...................
endif
endif
set yes = $no

nawk 'BEGIN ( rightday = 0 ) $2 ~ /'$month'/ && $3 ~ /'$yes'/ { rightday = 1; print } /error/ && rightday == 1 {print}' alert_mes.log

Then I got the answers:
Mon Sep 30 02:02:23
Mon Sep 30 09:31:21
ORA014:error 34512 in xxjob...
Mon Sep 30 12:31:21
ORA014:error 34512 in xxjob...
Mon Sep 30 22:02:23

Q:I don't know how to write it that just only print error line and date. That's to say I want the following answer:

Mon Sep 30 09:31:21
ORA014:error 34512 in xxjob...
Mon Sep 30 12:31:21
ORA014:error 34512 in xxjob...

Thank you very much!

piggy213

 

[Daedelus]

Well, first I have to admit I was not as smart as I thought! I wrote my reply at home, where I do not have a UNIX box handy, so I didn't try my program out. I did not discover I had messed up Carlos Almeida's date trick. I thought TZ was just a number you needed to add 24 to get yesterday's date. This is not true: TZ is a string which "date" interprets, so what you need to do is append "+24" to the string:

set TZ="$TZ+24"

In fact, the following works for csh:

env TZ="$TZ+24" date +"%a %b %e"

It returns (today, Oct 1st)

Mon Sep 30

And it has the advantage that the change to TZ is only in effect for the one date call. Entering

env TZ="$TZ+24" date +"%a %b %e" ; date +"%a %b %e"

results in:

Mon Sep 30
Tue Oct  1

(if you need 01 instead of 1 for today's date, use %d instead of %e). So you can replace your date finding instructions with

set yestermonth=`env TZ="$TZ+24" date +%b`
set yesterday=`env TZ="$TZ+24" date +%e`
set tomonth=`date +%b`
set today=`date +%e`

(I have tried these in csh and they both work!)

As for the nawk command, try this:

nawk '
   BEGIN {date=""}
   $2 == '$yestermonth' && $3 == '$yesterday' {date=$0}
   $2 == '$tomonth' && $3 == '$today' {exit}
   /:error/ && date != "" {print date; print}
   ' alert_mes.log

Now it saves the date when found, instead of printing it out. When it finds an error, it will print out both the date and the error line. Also, I have placed a check for todays date back in. You showed in your first message that some entries for today are in the log. Your program will print today's error messages as well as yesterday's, but will only print yesterday's date. I do not think that is what you want!. Mine will stop looking as soon as it finds today's date, so it only shows you what happened yesterday.

 

[piggy213]

Hi Daedelus,
Thank you very much.
May I ask you questions?
1) Why you write date="" behind the BEGIN?

2) $2 == '$tomonth' && $3 == '$today' {exit} --> The line I don't understand. If the field number 2 and 3 is equal to today's month and date(Oct 1) then exit. Why? If other condition exists(except yesterday error) eg. Sep 25,does it still exit? Sorry, my english is poor. What I want is to print yesterday error line and date not all yesterday date in the log, just only error's date .
eg. alert_mes.log:
1 Mon Sep 30 02:09:33
2 ORA014:error 34512 in xxjob...
3 Mon Sep 30 22:02:23
4 ORA014:normal 23455.....

I want :
1 Mon Sep 30 02:09:33
2 ORA014:error 34512 in xxjob...

 

[Daedelus]

1) "date" is not a command inside the nawk program. I used it as a variable, and I set it to "" at the beginning to make sure it is empty at the start. I also used it as a flag (like I used "rightday" in the first). This is poor programming practice, but I learned to program in the old days when space was valuable, and still use the old tricks without thinking. (Try writing a good program when the operating system, program, data, and "working" space must all fit in 16K of memory!)

2) You said you wanted only errors from yesterday. If the program finds today's date, then it has found all errors from yesterday, so it stops looking and exits. Without this line, the program will print errors from today, but still give yesterday's date: on the file

Mon Sep 30 02:09:33
ORA012:error 12345 in xxjob...
ORA014:normal 23455.....
Tue Oct 1 03:29:43
ORA012:normal 23456....
ORA014:error 34512 in xxjob...

the program would give

Mon Sep 30 02:09:33
ORA012:error 12345 in xxjob...
Mon Sep 30 02:09:33
ORA014:error 34512 in xxjob...

which is wrong! (Your program would do the same except it will not print the second date.) The {exit} line causes the program to stop as soon as it finds Tue Oct 1, so it gives

Mon Sep 30 02:09:33
ORA012:error 12345 in xxjob...

I am assuming your log is ordered by date, so that stuff for today will only occur after all stuff from yesterday.

3)"yesterday" means only the day before today, not earlier days. The program compares dates against yesterday's date. It will not print anything until it finds yesterday's date. Then it saves the line with the date and looks for lines with ":error". When it finds one, it will print the line with the date that it saved, and then the line with the error. If it finds another line with yesterday's date, it will save that line in place of the first, and print it when it finds an error. It does not print a date for anything other than error lines, and only when the date of the error is yesterday. When it finds today's date, it knows it has found everything from yesterday, so it quits.

Try this rewrite. I have changed the variable names to clear up the confusion, and added a separate flag for the date check. Also, the double quotes around the shell variables will prevent a format error that occurs for dates between 0 and 9 (such as today's date!):

#!/bin/csh -f

set yestermonth=`env TZ="$TZ+24" date +%b`  #=month for yesterday
set yesterday=`env TZ="$TZ+24" date +%e`    #=day number for yesterday
set tomonth=`date +%b`                      #=month for today
set today=`date +%e`                        #=day number for today

nawk '
   BEGIN {rightday=0; dateline=""}  # initialize variables
#        if date = yesterday, set flag, save date   
   $2 == '"$yestermonth"' && $3 == '"$yesterday"'  {rightday=1; dateline=$0}
#        if date = today, we are done: exit.
   $2 == '"$tomonth"' && $3 == '"$today"' {exit}
#        if error yesterday, print date, print error line 
   rightday == 1 && /:error/ {print dateline; print}
   ' alert_mes.log

 

[piggy213]

Hi Daedelus,
Today I tried the same program which you wrote, it could't work. Not all couldn't execute, just the date condition has something wrong. If I used the program you wrote, it show me the following messages:

env: No such file or directory
env: No such file or directory

Then I tested command in csh environment.

%setenv TZ $TZ+24

%date
Tue Oct 1 09:08:25 ROC 2002

It's right. I got the yesterday's date.

In C shell if I set environment variable it can't use "=". But I don't know how to set it in C shell program. So I still used my silly way to solve date problem and combined your nawk parts, it worked!

I extremly appreciated your helps recently.
Thank you very much.

piggy

 

[Daedelus]

Obviously, you have a different version of the C shell than I do, since those lines worked for me.
I have been trying to steer you away from using the method you showed for getting yesterday's date because unless you have changed the script from what is here, it will not work on the first day of any month other than October!
I was going to tell you to simply put the

setenv TZ $TZ+24
[\code]
line before the date calls for yestermonth and yesterday, but while I was experimenting with it, I found out that this timezone trick has deep flaws! For most of us, it will not work all of the time. So --- scrap that means of finding yesterday's date. Instead we need to make your method work for all days. Try this:

set today=`date +%e`
set tomonth=`date +%b`
@ yesterday = $today - 1
if ( $yesterday == 0 ) then
   if ( $tomonth == Jan ) then 
       set yestermonth=Dec
       set yesterday=31
   endif
   if ( $tomonth == Feb ) then 
       set yestermonth=Jan
       set yesterday=31
   endif
   if ( $tomonth == Mar ) then
       @ leapyear = `date +%C` % 4
       set yestermonth=Feb
       set yesterday=28
       if ( $leapyear == 0 ) yesterday=29
   endif
   ...
else
   set yestermonth=$tomonth
endif

This would be easier with a "switch" command, but I cannot get the syntax right.

 

[piggy213]

Hi Daedelus:
This is my program:

#!/bin/csh -f
set month = `date +%b`
set day = `date +%e`
@ yesterday = $day - 1
set yes = $yesterday

if ($yes == 0) then
if ($month == Jan) then
set month = Dec
set yes = 31
endif
if ($month == Feb) then
set month = Jan
set yes = 31
endif
.........................
.........................
if ($month == Mar) then
@ leapyear = `date +%C` %4
set month = Feb
set month = Feb
set yes = 28
if ($leapyear == 0) yes = 29
endif
else
set yestermonth = $month
endif

nawk 'BEGIN { rightday = 0; dateline = "" } $2~~ /'$month'/ && $3 ~~ /'$yes'/ { rightday = 1; dateline = $0 }$2~~ /'$month'/ && $3 ~~ /'$day'/ {exit} rightday == 1 && /ORA/ {print dateline;print}' alert_mes.log

Then I got :

Wed Oct 2 18:25:16 2002
ORA-12012: error on auto execute of job 134
Wed Oct 2 18:25:16 2002
ORA-12541: TNS:no listener
Wed Oct 2 18:25:16 2002
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 794
Wed Oct 2 18:25:16 2002
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 851

But I wanted :

Wed Oct 2 18:25:16 2002
Errors in file /raid1/app/oracle/admin/mes/bdump/mes_j000_5985.trc:
ORA-12012: error on auto execute of job 150
ORA-12541: TNS:no listener
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 794
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 851

The original file is :
italic
................................................
.................................................
Wed Oct 2 18:25:16 2002
normal xxxxxxxxxxxxxxxxxxxx
Wed Oct 2 18:25:16 2002
Errors in file /raid1/app/oracle/admin/mes/bdump/mes_j000_5985.trc:
ORA-12012: error on auto execute of job 150
ORA-12541: TNS:no listener
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 794
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 851
................................................
That is to say if it meet the "ORA" words I wanted to print all lines which are below the date.
Today I modified my program again and again. But it failed.
Could you help me?
Thank you very much.
piggy