Need help to parse and summarize a log file

[mutasie]

Hey guys,

Perl newbie here. I need some advice.

Here is the senario: I have a log file that looks like the following:

06/19 16:14:47: ERROR: error type 1
06/20 17:25:48: ERROR: error type 1
06/20 18:56:52: ERROR: error type 1
06/21 04:00:15: ERROR: error type 1
06/22 00:03:02: ERROR: error type 1
06/22 15:13:39: ERROR: error type 1
06/22 15:15:40: ERROR: error type 1

I want to write up a scirpt to summarize the above log file into the following format:

[date] [# of errors]
06/19 1
06/20 2
06/21 1
06/22 4

The date part is easy to get, but I can't figure out how to get the count of errrors. Any advice?

 

[KevinADC]

use a hash where the dates are the hash keys and the value is the count.



------------------------------------------
- Kevin, perl coder unexceptional!

 

[stevexff]

Something like

use strict;
use warnings;

my %summary;

while (<>) {
   $summary{$1}++ if (/^(\d\d\/\d\d)/);
}

print "$1\t$summary{$1}\n" for (sort keys %summary);

(not tested)

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object:erlDesignPatterns)[/small]

 

[mutasie]

Thank you so much for the input. Adding one more level of complexity, if the input were the following:

06/19 16:14:47: ERROR: error type 1
06/20 17:25:48: ERROR: error type 1
06/20 18:56:52: ERROR: error type 2
06/21 04:00:15: ERROR: error type 1
06/22 00:03:02: ERROR: error type 1
06/22 15:13:39: ERROR: error type 2
06/22 15:15:40: ERROR: error type 1

And the output should look like:
[date] [type of error] [# of errors]
06/19 1 1
06/20 1 1
06/20 2 1
06/21 1 1
06/22 1 2
06/22 2 1

How should I handle this?

Thanks.

Allen

 

[KevinADC]

Keep adding hash keys and counting what you need. What have you tried?

------------------------------------------
- Kevin, perl coder unexceptional!

 

[mutasie]

Keven,

Thank you for the advice.

I have decided that I will actually parse out the strings and save them in a database. I can do SQL to analize the log.

Allen

 

[travs69]

That is like renting a huge truck to move a small box.. Perl can easily do the preprocessing (even if you want to store the end data in a database).

You just need to keep working on it.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;

 

[mutasie]

I know... still working on my hash skills.

A

 

[stevexff]

There seems to be a common failing among posters, where they ask for X. Then, after a suitable answer is given, they say "ah, that's all very well, but what I really wanted was X and Y".

mutasie, look at the sample I gave you, and consider how much extra effort it would be to put two sets of capturing parentheses in the regex, and just do

$summary{$1}[red]->{$2}[/red]++ if (/^(\d\d\/\d\d)[red].*\s+(\d+)$[/red]/);

instead. I'll leave it to you to figure out the print statement. You can always use Data:umper to help you to visualise the structure...

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object:erlDesignPatterns)[/small]