find records via date range?? 2

[mojo24]

Hi i am a noob and need a little help to finish my shell script. I am learning as i go but hit a problem.

I am search thorugh logs(*.rv) files to find entires between two user defined dates,ie dd mm yyyy - dd mm yyy

The script so far looks for the "START" and "END" of each entry at sees if it belongs To an ORGID number. then only take the whole record based on OGID and in the a date range given


gawk '/^START/ { ok=0; sec="" } /^START/,/^END/ { sec=sec "\n" $0 }$2==281688 { ok=1 }/^END/ && ok { print sec }' *.rv>BUSORG_ORGID${BUSORG_ORGID}st${START_DAY}${START_MONTH}end${END_DAY}${END_MONTH}.txt



extract of one *.rv file

START: Mon Apr 24 10:09:19 2006:
Mon Apr 24 10:09:19 2006: service: before tpcall(), buffer being sent...
G_FLAG 0
CLFY_SUB slicker
MESSAGE_ID fooo bar foo
BUSORG_ORGID 281688

service: now I'm sending that...
Mon Apr 24 10:09:19 2006: Mon Apr 24 10:09:20 2006: service: after tpcall(), buffer received...
E_NUMBER 2
G_FLAG 0
CLFY_SUB bla bla
E_STRING No record(s) found
MESSAGE_ID foo bar
BUSORG_ORGID 281688

service: now I have received that...
Mon Apr 24 10:09:20 2006: service: field [XFDTRANSACT_XID2] not found, Ferror 4, tperrno 0
processing ends... Mon Apr 24 10:09:20 2006:
END:
START: Mon Apr 24 10:09:53 2006:
Mon Apr 24 10:09:53 2006: service: before tpcall(), buffer being sent...
CLFY_SUB foobar
MESSAGE_ID 1273498170sa;dooif
BUSORG_ORGID 422345
XPRODUCT_XACCOUNTNO hummm bug
XPRODUCT_XCOINTYPE foobar
END:

 

[Annihilannic]

You haven't said what the problem is.

Annihilannic.

 

[mojo24]

Your right.
sorry
it the date thing i am not sure how to do it.

Sure how to compare dates and find if they are in a desired range.i.e

I thought i would convert start date and end date :

from dd mm yyy to yyy ddd

then minus end yyy ddd from start yyy ddd

then ???

I am not sure how to do this and how add it to my script

 

[Annihilannic]

You could convert the dates to a format that would allow you to compare them using > and <, e.g. convert "Mon Apr 24 10:09:20 2006" to "20060424100920". You would have to use a lookup table to convert the month names to 01, 02, ... 12.

Since your awk script will be quite long you might want to separate it over multiple lines for clarity, or put it into a separate file and use gawk -f yourawkscript *.rv>BUSORG_ORGID${BUSORG_ORGID}st${START_DAY}${START_MONTH}end${END_DAY}${END_MONTH}.txt.

Annihilannic.

 

[mrn]

You could use the perl module Date::Manip

change the start / end date to match your range

#!/usr/local/bin/perl
use Date::Manip;

$start="01-jan-2000";
$end="05-May-2006";

while (<>)
{
@a = split('\s+');
print if ( (Date_Cmp($a[0],$start) >= 0) &&
(Date_Cmp($a[0],$end) <= 0) );
}

Mike

"A foolproof method for sculpting an elephant: first, get a huge block of marble, then you chip away everything that doesn't look like an elephant."

http://www.airport-parking-site.co.uk/

 

[mojo24]

Thankyou both for your suggetions They sound good but a little confused on how to impliment them

Dear Mike

The code looks good however my perl knowledge is very basic. So i see that it will find the right range , but what about the ORGID?

How do I incoperate this into my script?

gawk '/^START/ { ok=0; sec="" } /^START/,/^END/ { sec=sec "\n" $0 }$2==281688 { ok=1 }/^END/ && ok { print sec }' *.rv>results.txt


mike.pl<results.txt ????

 

[Annihilannic]

I was thinking something roufghly like this (not tested very much):

gawk '
BEGIN {
        months["Jan"]="01"
        months["Feb"]="02"
        months["Mar"]="03"
        months["Apr"]="04"
        months["May"]="05"
        months["Jun"]="06"
        months["Jul"]="07"
        months["Aug"]="08"
        months["Sep"]="09"
        months["Oct"]="10"
        months["Nov"]="11"
        months["Dec"]="12"
}
function datenumeric(longdate) {
        split(longdate,a)
        gsub(":","",a[4])
        return a[5] months[a[2]] a[3] a[4]
}
/^START/ {
        ok=0; sec="";
        date=substr($0,7);
        sub(":$","",date);
        numeric=datenumeric(date)
}
/^START/,/^END/ { sec=sec "\n" $0 }
$2==281688 { ok=1 }
/^END/ && ok && numeric > "20060420000000" && numeric < "20060421235959" { print sec }' *.rv

Annihilannic.

 

[mojo24]

Thanks Annihilannic
thats great i still need to work on it a little i think but at least i have a good idea form your script.

I did test yours but got nothing in a out put file

/^END/ && ok && numeric > "20060420000000" && numeric < "20060421235959" { print sec }' *.rv>dateresults.txt

 

[Annihilannic]

If you're using the same test data as above try changing the end date to 20060425235959.

Annihilannic.

 

[mojo24]

thanks for the tip
i just tried :

/^END/ && ok && numeric > "20000000000000" && numeric < "20060425235959" { print sec }' *.rv>dateresults.txt

and still nothing

 

[Annihilannic]

I just tested it with the sample data in your original post on Solaris and Linux using gawk, awk, nawk, /usr/xpg4/bin/awk... all worked fine. Try sticking some debugging prints in there somewhere perhaps to see if it's calculating the dates properly, etc. i.e. copy the "return" line and change "return" to "print".

Annihilannic.

 

[mojo24]

great it works !!

a sysadmin changed the permissions on the file without telling me over the weekend. your tips have been great thank you very very much.

If I now wanted to pass user input to gwak script ie.

#!/bin/sh
echo "What is DSN or BUSORG_ORGID for your search?"
read BUSORG_ORGID #stores ORGID
echo "DSN number $BUSORG_ORGID. From what start date format 20 05 2006 ?"
read START_DAY START_MONTH START_YEAR #stores start Date
echo "cut off date for search format 20 06 2006?"
read END_DAY END_MONTH END_YEAR #stores end Date
echo "$END_DAY $END_MONTH $END_YEAR"
echo "I will create you a file called BUSORG_ORGID${BUSORG_ORGID}st${START_DAY}${START_MONTH}$end${END_DAY}${END_MONTH}"
gawk '
BEGIN {

 

[Annihilannic]

You can set variables to use in your awk script using the -v switch, see the man page...

Annihilannic.