Read a File Backwards? 1

[Guest_imported]

I have a very large log file (could be a megabyte)
which I want to get data from, but only the last
100 lines of data!

Is there a way to open the file backwards and read it?
I just want to read the last 100 lines of a very large file
without having to load the whole thing into memory, or search through it. There must be a way!

Thanks for any hints, tips, or info!

 

[goBoating]

If you are on a *nix system, a simple system call....

@lines = `tail -100 your_log_file`;

I don't know, if you're on a Win system, other that read the file into an array and grab the last 100 elements.

Hopefully, you're on a *nix system.

HTH

If you are new to Tek-Tips, please use descriptive titles, check the FAQs,
and beware the evil typo.

 

[paulfrance]

I don't think there are any built in mechanisms to do this but I'm sure its possible using some combination of the following:-

open the file
seek the end
use tell to get the current file position

If your records are fixed length deduct length x 100 in bytes from tell and then seek there and then read the lines from there.

If your records are not fixed length then you'll need to write a backwards line reading subroutine that reads the file backwards, starting from a postion you pass in, a byte at a time until it hits a new line (don't forget if its win32 then newline \r\n will become \n\r) and then returns the new position. You could also pass in the number of lines backwards you want to read.

Then read the file from the position returned.

Writing the code is left as an exercise for the reader

paulf

 

[paulfrance]

Thinking about it, don't read the file backwards a byte at a time, that would be too painful. If you don't know the length of a record, have a guess at the approximate size of 100 lines and seek back to there. Read the end of the file into a string and count the linebreaks. If there's less than you need then try again reading a bit more until you get the quantity you need. You can then tidy up the start of the string which will almost certainly contain a partial record and then split the string into records as you please.

paulf