Perl newbie here. I have some working code I strung up together with help from others along the way. I am still quite unclear of how I should approach this problem with using an array of hash, hash refs. Right now it works using hashes, but I was also advised I can use an array for thise too. As a newbie, I'm not sure which is better. The function of my script is fairly simple. I'm looking for best practices. Please advise and if you have any good examples I can use. Thank you!
Purpose of my script:
1) Mine through a server log file, get information on the date I put in either manually or systematically
2) For each date given, give me the backup set name
3) For every backup set name, grab either one of these variables if they appear (backup-size, backup-time, backup-status, or ERROR if its given for that backup set)
4) Generate a datafile with these values delimited in whatever format. This datafile will be used later as feed to another system.
5) Current issues:
My current script uses hashes, again from help. I'm still confused on hash of hashes and its use. I didn't go the array route, cause I'm not sure how to set that up.
6) Problems formatting the output in an ordered fashion
So, i'm looking at a structure like this:
server1:
MyDate (today's date)
--> MyBackupSet
--> Backup Attribute = Backup Value
server2:
MyDate (today's date)
--> MyBackupSet
--> Backup Attribute = Backup Value
...
Perl code:
Output from Dumper:
Sample log file used to create this datafile:
Format I would like to try and create if possible (datafile):
Purpose of my script:
1) Mine through a server log file, get information on the date I put in either manually or systematically
2) For each date given, give me the backup set name
3) For every backup set name, grab either one of these variables if they appear (backup-size, backup-time, backup-status, or ERROR if its given for that backup set)
4) Generate a datafile with these values delimited in whatever format. This datafile will be used later as feed to another system.
5) Current issues:
My current script uses hashes, again from help. I'm still confused on hash of hashes and its use. I didn't go the array route, cause I'm not sure how to set that up.
6) Problems formatting the output in an ordered fashion
So, i'm looking at a structure like this:
server1:
MyDate (today's date)
--> MyBackupSet
--> Backup Attribute = Backup Value
server2:
MyDate (today's date)
--> MyBackupSet
--> Backup Attribute = Backup Value
...
Perl code:
Code:
use strict;
use warnings;
use File::Basename;
use Data::Dumper;
my %MyItems;
my $ARGV ="/var/log/server1.log";
my $mon = 'Aug';
my $day = '06';
my $year = '2010';
while (my $line = <>)
{
chomp $line;
print "Line: $line\n" if debug;
if ($line =~ m/(.* $mon $day) \d{2}:\d{2}:\d{2} $year: ([^:]+):backup:/)
{
my $server = basename $ARGV, '.log';
my $BckupDate="$1 $year";
my $BckupSet =$2;
print "$BckupDate ($BckupSet): " if debug;
$MyItems{$server}{$BckupSet}->{'MyLogdate'} = $BckupDate;
$MyItems{$server}{$BckupSet}->{'MyDataset'} = $BckupSet;
$MyItems{$server}{$BckupSet}->{'MyHost'} = $server;
#$MyItems{$server}{$BckupSet}->{'MyServer'} = $server;
if ($line =~ m/(ERROR|backup-size|backup-time|backup-status)[:=](.+)/)
{
my $BckupKey=$1;
my $BckupVal=$2;
$MyItems{$server}{$BckupSet}->{$BckupKey} = $BckupVal;
print "$BckupKey=$BckupVal\n" if debug;
}
}
}
print Dumper(%MyItems);
Code:
$VAR1 = 'server1';
$VAR2 = {
'abc1.mil.mad' => {
'ERROR' => ' If you are sure is not running, please remove the file and restart ',
'MyLogdate' => 'Fri Aug 06 2010',
'MyHost' => 'server1',
'MyDataset' => 'abc1.mil.mad'
},
'abc2.cfl.mil.mad' => {
'backup-size' => '187.24 GB',
'MyLogdate' => 'Fri Aug 06 2010',
'MyHost' => 'server1',
'backup-status' => 'Backup succeeded',
'backup-time' => '01:54:27',
'MyDataset' => 'abc2.cfl.mil.mad'
},
'abc3.mil.mad' => {
'backup-size' => '46.07 GB',
'MyLogdate' => 'Fri Aug 06 2010',
'MyHost' => 'server1',
'backup-status' => 'Backup succeeded',
'backup-time' => '00:41:06',
'MyDataset' => 'abc3.mil.mad'
},
'abc4.mad_lvm' => {
'backup-size' => '422.99 GB',
'MyLogdate' => 'Fri Aug 06 2010',
'MyHost' => 'server1',
'backup-status' => 'Backup succeeded',
'backup-time' => '04:48:50',
'MyDataset' => 'abc4.mad_lvm'
}
};Sample log file used to create this datafile:
Code:
Fri Aug 06 00:00:04 2010: abc2.cfl.mil.mad:backup:INFO: START OF BACKUP
Fri Aug 06 00:00:04 2010: abc2.cfl.mil.mad:backup:INFO: PHASE START: Initialization
Fri Aug 06 00:00:04 2010: abc2.cfl.mil.mad:backup:WARNING: Binary logging is off.
Fri Aug 06 00:00:04 2010: abc2.cfl.mil.mad:backup:INFO: License check successful
Fri Aug 06 00:00:04 2010: abc2.cfl.mil.mad:backup:INFO: License check successful for lvm-snapshot.pl
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-set=abc2.cfl.mil.mad
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-date=20100806000004
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: abcsql-server-os=Linux/Unix
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-type=regular
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: host=abc2.cfl.mil.mad.melster.com
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-date-epoch=1281078004
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: retention-policy=3D
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: sql-abc-version=ZRM for MySQL Enterprise Edition - version 3.1
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: abcsql-version=5.1.32-Melster-SMP-log
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-directory=/home/backups/abc2.cfl.mil.mad/20100806000004
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-level=0
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: backup-mode=raw
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE END: Initialization
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE START: Running pre backup plugin
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE END: Running pre backup plugin
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE START: Flushing logs
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE END: Flushing logs
Fri Aug 06 00:00:05 2010: abc2.cfl.mil.mad:backup:INFO: PHASE START: Creating snapshot based backup
Fri Aug 06 00:00:10 2010: abc2.cfl.mil.mad:backup:INFO: Fri Aug 06 00:48:48 2010: abc4.mad_lvm:backup:INFO: raw-databases-snapshot=test abcsql sgl
Fri Aug 06 00:48:51 2010: abc4.mad_lvm:backup:INFO: PHASE END: Creating snapshot based backup
Fri Aug 06 00:48:51 2010: abc4.mad_lvm:backup:INFO: PHASE START: Calculating backup size & checksums
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: last-backup=/home/backups/abc4.mad_lvm/20100804200003
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: backup-size=422.99 GB
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: PHASE END: Calculating backup size & checksums
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: read-locks-time=00:00:04
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: flush-logs-time=00:00:00
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: backup-time=04:48:50
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: backup-status=Backup succeeded
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: Backup succeeded
Fri Aug 06 00:48:54 2010: abc4.mad_lvm:backup:INFO: PHASE START: Running post backup plugin
Fri Aug 06 00:48:55 2010: abc4.mad_lvm:backup:INFO: PHASE END: Running post backup plugin
Fri Aug 06 00:48:55 2010: abc4.mad_lvm:backup:INFO: PHASE START: Cleanup
Fri Aug 06 00:48:55 2010: abc4.mad_lvm:backup:INFO: PHASE END: Cleanup
Fri Aug 06 00:48:55 2010: abc4.mad_lvm:backup:INFO: END OF BACKUPFormat I would like to try and create if possible (datafile):
Code:
MyHost=>server1;MyLogdate=>Fri Aug 06 2010;MyDataset=>abc2.cfl.mil.mad;backup-time=>Fri Aug 06 2010;backup-status=>Backup succeeded