Create daily log from multi-month log file 1

[spconway]

I have a db log file that I archive off every 6 months but I would like to capture daily events from it and then grep against it with certain problem keywords.

Should I use awk(and how) to pipe all lines between "Sun Sep 15" and "Mon Sep 16" to a tmp file?


Sun Sep 15 2002
01:00:04
01:00:04 Usr 62: Logout by on batch. (453)
01:00:05 BROKER 0: Shutdown request received from Admin Server. (8852)
01:00:05 BROKER 0: Begin normal shutdown (2248)

~~~~~~~~~~

Mon Sep 16 01:00:04 2002
01:00:04 Usr 62: Logout by on batch. (453)
01:00:05 BROKER 0: Shutdown request received from Admin Server. (8852)
01:00:05 BROKER 0: Begin normal shutdown (2248)


Thanks
spconway@comcast.net

 

[marsd]

Sed is better for this.
Try:
sed -n '/Sep 15/, /Sep 16/p'

 

[Grant]

Hi guys,

Actually, the awk syntax would be almost identical:

awk '/Sep 15/,/Sep 16/' mylog


Grant.

 

[Grant]

Hi again,

By the way, I am guessing that:
a) You would like a script to do this, and
b) You don't actually want the second date (Sep 16) to show up.

You might try playing with the following script:

% cat my.awk
#!/usr/bin/awk -f

d1{
do
{
if($0 ~ d2) { exit; }
if(last) { print last; }
last=$0;
}while ( ( getline ) > 0 )
}


To run it:

% my.awk d1="Sep 15" d2="Sep 16" mylog
Sun Sep 15 2002
01:00:04
01:00:04 Usr 62: Logout by on batch. (453)
01:00:05 BROKER 0: Shutdown request received from Admin Server. (8852)
01:00:05 BROKER 0: Begin normal shutdown (2248)
~~~~~~~~~~

 

[Grant]

Hi guys,

Going back to the following syntax

awk '/Sep 15/,/Sep 16/' mylog

It worked very well, so I initially tried to create the equivalent script as follows:

The command line:
% my.awk d1="Sep 15" d2="Sep 16" mylog

The script:
#!/usr/bin/awk -f

d1,d2


I thought that looked cool. Unfortunately, it didn't work properly: the output included data after the 'Sep 16' line. (BTW, mylog contains the original example above.)


Does anybody have an idea why the script didn't work?

Grant.

 

[marsd]

No, That looks like it should work Grant.
Did you try substitution inside awk?
say:

BEGIN {
d1 = sprintf("%s", "Sep 15&quot ; d2 = sprintf("%s", "SEP 16&quot
}
???, Haven't tried it..

 

[Grant]

Hi marsd,

Nice thought. I tried that too, and it still fails:

The code:

% cat test.awk
#!/usr/bin/awk -f

BEGIN {
d1 = sprintf("%s", "Sep 15"
d2 = sprintf("%s", "SEP 16"
}

d1,d2



The command line:

% test.awk d1='Sep 15' d2='Sep 16' mylog


The result:

All lines print!


Say, if I found a bug in awk, do I win a prize?

Grant.

 

[vgersh99]

$0 ~ d1, $0 ~ d2

FYI:

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&th=fc5c672028f50390&rnum=22

vlad
+---------------------------+
|#include<disclaimer.h> |
+---------------------------+

 

[marsd]

So awk is not even looking at $0
unless forced with this pattern?

I don't use the pat , pat method
often in awk but that seems a little
strange..

 

[spconway]

First, thanks very much for replying.
I don't mind if Sep 16 appears.
I was thinking that I could run this script at 11:59pm everyday in cron. If I did run this awk script near midnight, would it be easier just to pipe out to another file from &quot;Sep 15&quot; to the end of the log file? Thanks
spconway@comcast.net

 

[Grant]

Hi vgersh99,

Works like a charm. Nice!

Now, what about my prize?

Grant.

 

[Grant]

Hi guys,

I just had a brainstorm! I think I understand why the notation $0 ~ re-variable is necessary.

Although awk can use the contents of a variable in the same places where the /regular expression/ syntax is used, it would think it was supposed to do an existence test on the variable if the $0 ~ re-variable notation was not used.

So, an awk program that looks like

#!/usr/bin/awk -f

d1

will print all lines if d1 is any none-null value, and no lines if null.


It seems that

#!/usr/bin/awk -f

d1,d2

has pretty much the same behaviour, but in this case the d2 seems to get ignored.

Damn! There goes my Nobel Prize.

Grant.

 

[vgersh99]

I don't quite remember all the specifics why this construct works in this case, but I think [as far as I can remember] it has to do with dynamic regex static.

It's been awhile since I read this lengthy discussion on comp.lang.awk - I wish I saved this thread. vlad
+---------------------------+
|#include<disclaimer.h> |
+---------------------------+

 

[spconway]

Thanks you all for the much appreciated help !!!

I created two awk scripts to pull out the daily from the monthly log. I will then run greps against the daily on key words and send email notification if an &quot;alert&quot; word is detected. This will all run in cron.(hopefully)

#!/bin/ksh
#NAME: my.awk
#
#
integer y=1 # is 1
integer z # is declared
M=`date +%b` # is abbrev current month
D=`date +%d` # is day
((z= $D + y)) # is today plus 1
cd /test/db
my2.awk d1=&quot;$M $D&quot; d2=&quot;$M $z&quot; test.lg > testawk.out

-------------------------------------------------

#!/usr/bin/awk -f
#NAME: my2.awk

$0~d1,$0~d2 Thanks
spconway@comcast.net