How to parse file from a point onwards

[bhaswar]

Hi,

I need help in writing a shell (awk / sed / shell). I am trying to write a script to be able to automate my daily check of the syslog and oracle alert logs.

Basically, I need to be able to extract the file from a point onwards based on date. So lets say today is 4 th Sept, I want to extract the file from 3rd Sep. Need some ideas on how to do this.

Thanks in advance.

Bhas

 

[columb]

The easiest answer is not to extract from the log file but to housekeep it. What I do is run a cron job which takes these log files and renames them to, for example, /var/log/syslog.060904.

Then, if I want to extract data from the last two days worth I can, for example

grep error syslog.06090[2-4]

If you want to you can combine the reporing and the housekeeping. For example,

report_on_syslog.ksh | mail -s "Syslog report" me@myserver
#syslog contains details from last 24 hrs
mv /var/log/syslog /var/log/syslog.$(date +%y%m%d)
#housekeep syslog
touch /var/log/syslog
#create a new one
find /var/log -name "syslog*" -mtime +30 -exec rm {} \;
#housekeep old syslogs

Then it is a matter of writing report_on_syslog.ksh to extract the details you want.

Ceci n'est pas une signature
Columb Healy

 

[p5wizard]

Here's a way to canonicalize an orcale Alert Log file (you may need to adjust the code for date strings in another language):

awk 'BEGIN{
 mon["Jan"]="01"; mon["Feb"]="02"; mon["Mar"]="03"; mon["Apr"]="04"
 mon["Jun"]="05"; mon["Jul"]="06"; mon["Aug"]="07"; mon["Sep"]="08"
 mon["Nov"]="09"; mon["Oct"]="10"; mon["Nov"]="11"; mon["Dec"]="12"
 datetime=""
}
{
 if ($0~/^[MTWFS][ouehra][neduit] [JFMASOND][aepuco][nbrylgptvc] [ 0-9]/) {
  day=substr($0,9,2)
  month=mon[substr($0,5,3)]
  year=substr($0,21,4)
  time=substr($0,12,8)
  datetime=sprintf("%04d/%02d/%02d %s",year,month,day,time)
 }
 else {
  if (datetime!="") {
   printf "%s %s\n", datetime, $0
  }
 }
}' /path/to/your/alert_XXX.log

output is similar to:

...
2006/08/04 11:51:45 Thread 1 advanced to log sequence 1002
2006/08/04 11:51:45 Current log# 2 seq# 1002 mem# 0: /oracle/XXX/origlogB/log_g12m1.dbf
2006/08/04 11:51:45 Current log# 2 seq# 1002 mem# 1: /oracle/XXX/mirrlogB/log_g12m2.dbf
2006/08/04 11:51:46 ARC1: Evaluating archive log 4 thread 1 sequence 1001
2006/08/04 11:51:46 ARC1: Beginning to archive log 4 thread 1 sequence 1001
2006/08/04 11:51:46 Creating archive destination LOG_ARCHIVE_DEST_1: '/oracle/XXX/oraarch/XXXarch1_1001.dbf'
2006/08/04 11:52:00 ARC1: Completed archiving log 4 thread 1 sequence 1001
...

This way it is easier to grep for entries of a specific date.

HTH,

p5wizard

 

[bhaswar]

Guys thanks for your replies. p5wizard, thanks for your helpful tip. that would have probably been my next question. :-D

I also found another useful one liner, which I am going to use:

tail +`grep -n "keyword" file | cut -d: -f1` file