Read a given line and the next line from a file

[kobewins]

Hi,

Please help me on this script. I need to read a given line and the next line (if applies) from a file, based the line number given in the other file. (more details are included in the description of the code)

I have got it work, but very slowly on large size of data. I knew the problem was the open/close file actions, but didn't know how to improve it.

Any advices are greatly appreciated.

Thanks
David

#!/usr/bin/perl 


######################################################################
#
# Author:	
# Date:		December 21, 2005
# Job Name:	Reloads.pl
# Usage: 	Reloads.pl loadPractition.rpt loadPractition.in	
# Output:	newLoads.dat
# 
# Job description:
#	Using the following File infile1 (loadPractition.rpt) to get the line 
#	number ($ln) for File infile2 (loadPractition.in), ie, the first number 
#	after 'Error Text:  ' in infile2. In my example, $ln will be 1, and 5. 
#	Then, loop through infile2, read the line ($ln) into an output file 
#	(newLoads.dat). if the next line ($ln+1) begins with 'A', write 
#	Line $ln+1 into newLoads.dat, too.
#
#### Input File 1: loadPractition.rpt
#
#LOAD0007                                           
#RUN DATE: Dec 13, 2005 12:07:42                    
#                                                        
#                         PRIMARY KEY                    
#                                       
#     Error Text:  1, , 12, DOCDEA,  , PRACTITIONER_REGI ... 
#     Error Text:  3, , 12, DOCDEA1,  , PRACTITIONER_REG ...                                                      
#                                                        
#RECORDS READ:          6     ACCEPTED:          2   
#
####  Input File 2: loadPractition.in
#
#PCDOCFIRSTNAME   DOCLASTNAME         DOCDEA      12PRE194001013000010
#AISTATELICID     082005120130000101
#PCDOCFIRSTNAME1  DOCLASTNAME1        DOCDEA1     32PRE195001013000010
#AISTATELICID1    082005120130000101
#PIDOCFIRSTNAME   DOCLASTNAME         STATELICID  58PRE194001013000010
#PIDOCFIRSTNAME1  DOCLASTNAME1        STATELICID1 68PRE195001013000010
#
####  Output File: newLoads.dat
#
#PCDOCFIRSTNAME   DOCLASTNAME         DOCDEA      12PRE194001013000010
#AISTATELICID     082005120130000101
#PIDOCFIRSTNAME   DOCLASTNAME         STATELICID  58PRE194001013000010
#
######################################################################

@ARGV == 2 or
	die "Usage: Reloads loadPractition.rpt loadPractition.in";
	
($rpt, $in) = @ARGV;

# Extract the needed lines, which contains 'Error Text:'
my @list = `grep "Error Text:" $rpt`;
print (@list); # contains: 1, 3, ...

# Get the number of lines in the $in file
my $total_ln = `wc -l < $in`;
warn "wc failed: $?" if $?;
chomp($total_ln);
$total_ln =~ s/^\s*//; #get rid of leading spaces
print "total_ln = $total_ln \n";

open(OUTFILE, "> newLoads.dat")
	or die "Can't open newLoad.dat for writing: $!\n";
	
foreach(@list)
{
	/Error Text:\s*(\d+)/;
	my $ln = $1;
	print "Line no. $ln\n";
	
	my $line;
	my $new_ln = 0;
	my $next_line;
    
	open(INFILE, "< $in") or die "Can't open file for reading: $!\n";
	while (<INFILE>) 
	{

		$line = $_;
		last if $. == $ln; # read a particular line into $line   
		
	}
	if ($. != $ln) 
	{
	    die "Didn't find line $ln in input file: loadPracNew.in\n";
	}
	#print "line: $line \n";	#write that line ($line) into a new ourput file	
	print OUTFILE $line;
	print "Dollar . = <$.> \n";
	close(INFILE);	
	
	# process the next line
	if ( $. <= $total_ln ) {
		$new_ln = $ln + 1;
		print "new_ln = <$new_ln> \n";
		open(IN, "< $in") or die "Can't open file for reading: $!\n";
		while (<IN>) 
		{
        	
			$next_line = $_;
			last if $. == $new_ln;
		}
		if(/^A/)
		{
			#print "next_line: $next_line \n";
			print OUTFILE $next_line;
			print "-- Dollar . = <$.> \n";	
		}
		close(IN);	
	}
	
}
 
close(OUTFILE);

 

[KevinADC]

is this school/class work?

 

[kobewins]

I've been out of school a long long time.

David

 

[MikeLacey]

Hello David,

Without having gone through your program in detail the thing that springs out at me is that you're processing the whole input file using the external grep utility. Can I suggest you consider the following approach? (error handling and stuff ommitted here for brevity)

open(INFILE,$rpt);
while(<INFILE>){
[tab]if(/Error text:/){
[tab][tab]# do something with the Error line here
[tab]}
}
close INFILE;

Mike

I am not inscrutable.

Want great answers to your Tek-Tips questions? Have a look at faq219-2884

 

[kobewins]

Thanks, Mike, for your suggestions.

My real and big problem is:

For eachline in the list (List L) or file (File B)
{
open and close File A for get one line, using the "$." variable;
then open and close the File A again for getting Line "$. + 1")
}

These open and close cost too much for the processing time.

I am looking for a way to enhence the performance.

--
David

 

[MikeLacey]

Ok, I see. You can get both lines without opening and closing with the trick below.

$ln = 2376; # for instance
# so this code will get two lines, 2376 and the
# next one, 2377

while(<INFILE>){
[tab]if($. == $ln){
[tab][tab]$first_line = $_;
[tab][tab]$second_line = $_ if $_=<INFILE>); # if not eof
[tab][tab]last;
[tab]}
}

Am I on the right track now? (sorry, bit busy and speed reading your code here)

Mike

I am not inscrutable.

Want great answers to your Tek-Tips questions? Have a look at faq219-2884

 

[kobewins]

Works perfectly. Many thanks, Mike.

-
David

 

[MikeLacey]

My pleasure

Mike

I am not inscrutable.

Want great answers to your Tek-Tips questions? Have a look at faq219-2884

 

[KevinADC]

left out one condition that was stated in the original post:

if the next line ($ln+1) begins with 'A', write
Line $ln+1 into newLoads.dat, too.

 

[MikeLacey]

Yes, the OP understands if() though; it was how to explicitly read another line from a file he needed help with.

I'm not here to write scripts for people you know I have my sleep to catch up on and my motorbike to ride!

Mike

I am not inscrutable.

Want great answers to your Tek-Tips questions? Have a look at faq219-2884

 

[KevinADC]

Yes, but surely adding the check for 'A' would have not added more than a minute to the work you already did, but I understand wanting to maximize sleep/bike time, even if it's just to gain an extra minute. Maybe it's not important, the OP doesn't have a check for it in their code either!

 

[kobewins]

Thanks.

Mike is right. I have added the condition to check 'A" for the the next line, after reading Mike's answer.

Again, thanks for all your advices and responses.


David