Problem with a time hash

[carg1]

Okay so I'm still wrestling with these log files. What I'm trying to do now is get it to go through and add up how long a user stayed on a page. The log is formatted like this:

19.33.44.63 2003/08/05 0009:38:11 PASSED

http://www.slashdot.com

110.25.12.6 2003/08/05 0009:38:11 PASSED

http://us.yimg.yahoo.com

101.59.34.99 2003/08/05 0010:42:53 PASSED

http://www.polo.com

210.6.13.3 2003/08/05 0010:42:54 PASSED

http://www.bolt.com

The above times are the real times from the log. Now I try running this script on it (my first actual script since finishing Perl for Dummies, be gentle, lol):

#Declare and localize the variables
my %Time;
my $User;
my $Dd;
my $Tt;
my $Ap;
my $Dest;
my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst);

#Open the text file
open(INDB, "d:\\perl\\test.txt") or die "Can't do it.";

#Begin a loop
while(<INDB>) {
	time;
	$Line = $_;
	($User, $Dd, $Tt, $Ap, $Dest) = split(/\s/, $Line);
	($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime($Tt);
	$Elapsed = ($hour, $min, $sec);
		if(exists $Time{$Elapsed}) {
			$Time{$Elapsed} += 1;
		}#End of if
			else {$Time{$Elapsed} = 1}#End of else
}#End of while

foreach $key( %Time) {
	print &quot;Total time elapsed is $key. \n&quot;;
}

I think I'm 50/50 on the right track, but I know something is wrong (other than just by looking at it) because I get this for output:

Total time elapsed is 10.
Total time elapsed is 2.
Total time elapsed is 9.
Total time elapsed is 2.

I welcome any and all help with this

 

[tonykent]

Am I correct in thinking that what you are trying to produce is the difference between the times listed in consecutive lines of the logfile? If that assumption is correct, I don't follow what you are trying to do after line "($User, $Dd, $Tt, $Ap, $Dest) = split(/\s/, $Line);"

In the way you use 'localtime' you are obtaining the current time when you run the script - that's nothing to do with what's in the logfile. Maybe I am misinterpreting what you are trying to do?

Anyway, if you are trying to obtain the time difference between the time listed in lines 1 and 2, 2 and 3 etc, try the following:

#!/usr/contrib/bin/perl
#Declare and localize the variables
use Date::Calc qw(Delta_DHMS);
my $User;
my $Dd;
my $Tt;
my $Ap;
my $Dest;
#Open the text file
open(INDB, "c:\\temp\\test.txt&quot or die "Can't do it.";
$i=1;
@array = <INDB>;
foreach $line(@array) {
@fields=split(/\s+/,$line); #split the line
($year, $month, $day) = split(/\//, $fields[1]);
($hh, $mm, $ss) = split(/:/, $fields[2]);
$logfile[$i] = { #write the data into record
&quot;User&quot; => &quot;$fields[0]&quot;,
&quot;Dt&quot; => [ &quot;$year&quot;,&quot;$month&quot;, &quot;$day&quot; ],
&quot;Tt&quot; => [ &quot;$hh&quot;,&quot;$mm&quot;, &quot;$ss&quot; ],
&quot;Ap&quot; => &quot;$fields[3]&quot;,
&quot;Dest&quot; => &quot;$fields[4]&quot;
};
$i++;
}
for ($a=1;$a<$i-1;$a++) {
@time1=(%{$logfile[$a]}->{&quot;Dt&quot;}[0], %{$logfile[$a]}->{&quot;Dt&quot;}[1], %{$logfile[$a]}->{&quot;Dt&quot;}[2], %{$logfile[$a]}->{&quot;Tt&quot;}[0], %{$logfile[$a]}->{&quot;Tt&quot;}[1], %{$logfile[$a]}->{&quot;Tt&quot;}[2] );
$b=$a+1;
@time2=(%{$logfile[$b]}->{&quot;Dt&quot;}[0], %{$logfile[$b]}->{&quot;Dt&quot;}[1], %{$logfile[$b]}->{&quot;Dt&quot;}[2], %{$logfile[$b]}->{&quot;Tt&quot;}[0], %{$logfile[$b]}->{&quot;Tt&quot;}[1], %{$logfile[$b]}->{&quot;Tt&quot;}[2] );
@diff = Delta_DHMS(@time1, @time2);
print &quot;time difference: $diff[0] days, $diff[1]:$diff[2]:$diff[3]\n&quot;;

}

 

[carg1]

Oookay, I see I was far overstepping my knowledge. Here's my original goals in pseudocode:

Ask which file you want to run time calculations on
Open that file
Begin an infinite loop
Associate the user with the time
Associate the time with the site
Calculate the time since the epoch of the first time value in the file
When a record is discovered, add the time value to the above time since the epoch
	If the time value already exists, count it as 1 instance of that time value
If the time doesn't exist, add its time in seconds to the running count
When a site is discovered, add it to the running count
	If the time and site value pair already exist, count it as 1 instance
If the site doesn't exist, add it to the running count

Run the localtime function to get the days, hours, minutes and seconds

Print how many records were found
Print the keys in this fashion: 
"User 'x' spent dD:HH:MM:SS total on 'a' page"
"User 'x' spent dD:HH:MM:SS total on 'b' page"
"User 'x' spent dD:HH:MM:SS total on 'c' page", etc.
End the program

To be a little more specific, I wanted it to consider the site by its domain name instead of the entire address, lest it return a hit for each different address in the same domain. Also, I wanted to set a threshhold of say, anywhere from 5-10 minutes (yet to be decided) that the script would consider as user inactivity. I wanted to do it manually, for the experience, but I did tinker with the Date::Calc module a bit. I couldn't really get it to work exactly how I wanted it to. I just tested your script and at first it gave me the messages &quot;Using a hash as a reference is deprecated at test.pl line 41.&quot; and &quot;Using a hash as a reference is deprecated at test.pl line 43.&quot; six times each. It did calculate the time correctly. The problem came when I tried it on the actual log file which has a few hundred entries. It gave me the abovementioned error message, in the same fashion. It then repeatedly, I don't know how many times, hundreds probably, returned &quot;Use of uninitialized value in string at test.pl line 34, <INDB> line 1820.&quot; Definitely far closer to the goal than mine came, but still a little off.

 

[tonykent]

I must be being dense but I'm still uncertain exactly how you want to manipulate the times in the log file. Are you after the difference between the time in line 1 and the time in line 2 (then between line 2 and line 3 etc) or do you want the time difference between that listed in line x and the current time (i.e. when the script is run)?

One thing to remember is that the format of the time obtained differs depending upon how you obtain it. My script above assumed the format taken from your log file i.e. 2003/08/05 0009:38:11 and used a method from Date::Calc that allows dates in this format (when split up) to be subtracted.

If you are obtaining times using localtime you must make sure that either the time is then translated into the same format as you have in your log file, or the log file format must be translated in epoch seconds, before any calculations are performed.

I don't understand why you got all those error messages when running my script: it runs without error on my (NT) machine when using your small log file example.

Sorry I can't be of more practical assistance!