PCAP reader to mysql

[jboy4]

Hi i am just learning perl for a company project they want me to make a pcap reader that puts the information into mysql.

I looked into what i would need and realized that tcpdumplog would be good for reading the pcap file but i am not sure where to go with the code to get it to save in record format to mysql.

So correct me if im not on the right track but this is what I was working with!

#!/usr/bin/perl

use Net::TcpDumpLog;

my $log = Net::TcpDumpLog->new();
$log->read("C:\Documents and Settings\jordant\My Documents\pcap\m1.pcap");


I still get an error with the above but wont that read my m1.pcap file? How do i get it to mysql format?

Any help is appreciated!

 

[jboy4]

Ok so i did some more research and found a cleaner code with more lables for easier identifcation.

#!/usr/bin/perl

use Net::TcpDumpLog;
use NetPacket::Ethernet;
use NetPacket::IP;
use NetPacket::TCP;
use strict;
use warnings;

my $log = Net::TcpDumpLog->new();
$log->read("/Dump/m1.pcap");

foreach my $index ($log->indexes) {
my ($length_orig, $length_incl, $drops, $secs, $msecs) = $log->header($index);
my $data = $log->data($index);

my $eth_obj = NetPacket::Ethernet->decode($data);
next unless $eth_obj->{type} == NetPacket::Ethernet::ETH_TYPE_IP;

my $ip_obj = NetPacket::IP->decode($eth_obj->{data});
next unless $ip_obj->{proto} == NetPacket::IP::IP_PROTO_TCP;

my $tcp_obj = NetPacket::TCP->decode($ip_obj->{data});
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);
print sprintf("%02d-%02d %02d:%02d:%02d.%d",
$mon, $mday, $hour, $min, $sec, $msecs),
" ", $eth_obj->{src_mac}, " -> ",
$eth_obj->{dest_mac}, "\n";
print "\t", $ip_obj->{src_ip}, ":", $tcp_obj->{src_port},
" -> ",
$ip_obj->{dest_ip}, ":", $tcp_obj->{dest_port}, "\n";
}


My issue is $log->read. seems to always not find my file. I have my file on the Desktop. I tried using a direct path to it but maybe perl needs a specific way? Sorry im new.

Also how can i get this to database into mysql?? Thanks again

 

[travs69]

$log->read("/Dump/m1.pcap");
does not look like it is on your desktop. You will have to deal with the spaces in windows files name, I suggest just keeping in a directory with no no spaces c:/blah/m1.pcap and reading it in like that "c:/blah/m1.pcap".


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;

 

[jboy4]

Ive got the Pcap reading out because i can use the print command to see it. But how would i get it into mysql?

#Login to mysql
$dbh = DBI->connect('DBI:mysql:test', 'root', 'password'
) || die "Could not connect to +database: $DBI::errstr";

#The Headers in Table
$dbh->do("CREATE TABLE test2 (Source , Destination VARCHAR(100), Packets VARCHAR(100))");

#Info in Table
$dbh->do( "INSERT INTO test2 (Source,Destination,Packets)
values ('$ip_obj','$tcp_obj','$eth_obj')");

So my question is the last part how would i produce my earlier tcpdump into the table???

Also how do you get TCPDUMPLOG to show the packets??

 

[jboy4]

2 things left to do on it.

Show output from pcap of time when packets were used.
Show output for Packets being used (in bytes)

 

[jboy4]

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);


Thats my time string but it seems to not work correctly.

I get the right month and day. But year and Mins/Sec are off.

 

[Annihilannic]

How far off? And for what values of $secs and $msecs?

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[jboy4]

Year is way off. sometimes gives the ecop 1969. Do you have a code to output time from the pcap when packets were taken?

I went another path and getting wrong dates as well.


use Time::HiRes qw(gettimeofday);

@HiResTime = gettimeofday();
($sec,$min,$hour,$day,$month,$year) = localtime($HiResTime[0]);


$month = $month + 1;
$year = $year + 1900;
my $time = sprintf "%02d/%02d/%4d %02d:%02d:%02d:%06d",
$month,$day,$year,$hour,$min,$sec,$HiResTime[1];

outputs: 111/05/211012 16:22:19:578125

Times are all the same and 111 goes up to 172.

05 all the same. 3rd number is 6 digits in 300,000 range.

 

[Annihilannic]

Please output the values of $secs and $msecs from your original code and let us know what exact (incorrect) values are returned by localtime().

For your second example, you haven't provided a complete list of values on the left hand side of localtime(). You can't just omit the ones you don't want because localtime() returns all of them anyway; by doing so you will end up with unexpected values in your variables. You can use undef placeholders if you'd rather not define variables for the unused values.

I don't think localtime() understands the HiResTime value you are supplying anyway... not sure about that?

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[jboy4]

Is there another value other then localtime to be used? I added all the variables for local time and the $month comes back as 1-172 counting up by 1. because of $month = $month + 1;

my year is insanley big 300,000. My time is just my current computer time.


I need something to grab the times from the pcap. Wireshark has it in Seconds. I cant even get that output in perl.

 

[Annihilannic]

Can you show us a sample of the timestamps returned by pcap? I don't have it installed so I have no idea what it looks like.

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[jboy4]

Ive got it working properly now. But here is my final question. If i have it reading a log file like so:

$log->read("C:\\Documents and Settings\\jordant\\Desktop\\Dump\\m1.pcap")

How could i get it so it reads within the Dump folder but all files that are .pcap?

i will run into problems putting each file in individually because we will be adding more i want it to read all files(that are pcap files)within the folder [C:\\Documents and Settings\\jordant\\Desktop\\Dump].

 

[Annihilannic]

Wrap a loop like this around it:

[COLOR=#006600]#!/usr/bin/perl -w[/color]
[COLOR=#0000FF]use[/color] strict;

[COLOR=#0000FF]foreach[/color] [COLOR=#0000FF]my[/color] $pcapfile ([COLOR=#FF0000]glob[/color] [COLOR=#808080]"C:\\Documents and Settings\\jordant\\Desktop\\Dump\\*.txt"[/color]) {
        $[COLOR=#FF0000]log[/color]->[COLOR=#FF0000]read[/color]($pcapfile)
        [COLOR=#006600]# ...[/color]
}

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[jboy4]

Alright so i tryed adding this into my code but for some reason i get an error.

Here is the exact message:
ERROR: Can't read log C:./Documents: No such file or directory.

The directory does exist tho. I can even put back the exact path with it going to that directory to m1.pcap and it works...?

Here is my entire code.

#!/usr/bin/perl -w


 use DBI;
 use Net::TcpDumpLog;
 use NetPacket::Ethernet;
 use NetPacket::IP;
 use NetPacket::TCP;
 use Net::Pcap;


 #Login to mysql
$dbh = DBI->connect('DBI:mysql:test', 'root', 'nstar'
	           ) || die "Could not connect to +database: $DBI::errstr";



 #Pcap file to log
 my $log = Net::TcpDumpLog->new();
 foreach my $pcapfile (glob "C:\\Documents and Settings\\jordant\\Desktop\\Dump\\*.pcap") {
        $log->read($pcapfile)
        }

 #INFO from PCAP file
foreach my $index ($log->indexes) {
  my ($length_orig, $length_incl, $drops, $secs, $msecs) = $log->header($index);
  my $data = $log->data($index);



  my $eth_obj = NetPacket::Ethernet->decode($data);
  next unless $eth_obj->{type} == NetPacket::Ethernet::ETH_TYPE_IP;


  my $ip_obj = NetPacket::IP->decode($eth_obj->{data});
  next unless $ip_obj->{proto} == NetPacket::IP::IP_PROTO_TCP;

  my $tcp_obj = NetPacket::TCP->decode($ip_obj->{data});





      #get date time stamp of packet
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);
    $mon+=1;
  my $time = sprintf("%02d-%02d %02d:%02d:%02d",
    $mon, $mday, $hour, $min, $sec);





  #Info in Table

  $dbh->do( "INSERT INTO  test2 (Date,Source,Destination,Packets,Port)
                        values (
                        '$time',
                        '$ip_obj->{src_ip}',
                        '$ip_obj->{dest_ip}',
                        '$ip_obj->{len}',
                        '$tcp_obj->{dest_port}')");


  }

 

[Annihilannic]

If you add a print message like this, does it look like it is trying to read the correct file names?

        print "Reading $pcapfile"
        $log->read($pcapfile)

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[inforeqd]

Sorry can't provide any help but I like the idea of what you are doing. When you get it all worked out can you post the complete code?? Thanks

 

[jboy4]

No Annihilannic. It errors out with no directory/or file like i said above. I worked on it for a little and i got it outputting cannont read m2.pcap because its not a file.?

Here what i changed it too.

    my $dir = 'C:\\Documents and Settings\\jordant\\Desktop\\Dump\\';

    opendir(DIR, $dir) or die $!;

    while (my $file = readdir(DIR)) {

        # We only want files
        next unless (-f "$dir/$file");

        # Use a regular expression to find files ending in .pcap
        next unless ($file =~ m/\.pcap$/);

     my $log = Net::TcpDumpLog->new();
      $log->read($file)
      }
       print $file;
      exit 0;