help - A better way?

[fidge]

Hello

I have the following which although works, runs very slowly. Can anybody help with a better/faster approach?
i'm trying to find out the concurrent rate of transactions

Thanks.

input file
==========
start date/time , end date/time
0610060000580,0610060001000
0610060000580,0610060001000
0610060000590,0610060001000
0610060001020,0610060001040
0610060001030,0610060001040
0610060001040,0610060001060
0610060001040,0610060001060
Script
=======
cc=1
oldstart=o
for REC in $(cat $1) ; do
start=$(echo ${REC} | cut -d"," -f1)
end=$(echo ${REC} | cut -d"," -f2)

if [ $start != $oldstart ]
then
cc=1
fi

if [ $start = $end -o $start = $oldstart ]
then
let cc=cc+1
fi
echo "Concurrent rate at $start = $cc" >> $1.res
oldstart=$(echo ${start})
done

 

[fidge]

Sorry wrong forum. Moved to unix scripting

 

[Annihilannic]

I would use awk for this:

awk -F, '
        BEGIN { cc=1 }
        $1 != oldstart { cc=1 }
        $1 == $2 || $1 == oldstart { cc=cc+1 }
        { print "Concurrent rate at " $1 " = " cc ;  oldstart=$1 }
' inputfile

Annihilannic.

 

[fidge]

Thanks, that worked in under 30 seconds for 90k records.

 

[olded]

It might be an interesting technical exercise to change the script and let the shell do the parsing:

#!/bin/ksh

cc=1
oldstart=o
while IFS="," read start end
do
        if      [ $start != $oldstart ]
        then
                cc=1
        fi

        if      [ $start = $end -o $start = $oldstart ]
        then
                let cc=cc+1
        fi
        echo "Concurrent rate at $start = $cc"  >> $1.res
        oldstart=$(echo ${start})
done < "$1"

 

[Annihilannic]

You might want to change this:

[tt]oldstart=$(echo ${start})[/tt]

to this:

[tt]oldstart=${start}[/tt]

I would also quote all of the variables in the tests to make it 'safer', e.g.

[tt][ "$start" = "$end" -o "$start" = "$oldstart" ][/tt]

Annihilannic.

 

[fidge]

Thanks olded and annihilannic, but i'm going to stick with the awk option. I need to run this against 30 files containing between 80-120k records, and awk is so much faster.
thnaks again.