Best method to read very large text files in Java (>1.5GB)

[bitwise]

I have a very large ASCII text file. It's an Apache webserver log. It contains 4,783,012 lines and is 1.66GB. I need to look at each line of the file independently. This makes using "java.nio" a bit more difficult because I'll have to parse out a line from a buffer by looping through the buffer looking for '\r' or '\n' to determine a line in the log.

Of course, I could do the standard...

FileReader fr = new FileReader("C:\access.log"); 
BufferedReader br = new BufferedReader(fr); 

String strLine;			 
while ((strLine = br.readLine()) != null) {
 	//PARSE LINE...
}

However, this takes 30 - 40 seconds just to do a line count. Is there a more efficient method I should look at to doing this?

Thanks!

 

[TonHu]

Is there a more efficient method I should look at to doing this?

Not do it in Java, but use existing tools to parse a webserver logfile. Several available after a Google search

 

[bitwise]

The question is not just about how to parse log files. There are tons of tools and scripts available to do that. That was more of an example to illustrate my question.

 

[Diancecht]

You can read data blocks instead of lines to reduce read operations. Take a look at this

Cheers,
Dian