Updating a large file from a second large file

[gsdcrazy]

This should be simple, but I need to update a master file from a second file. The master file has all records and more than the second file. The second file never contains records that are not in the first file. The master file contains 2 million records. The second file contains from 1 million upward to 2 million records. Neither file can be loaded into an array. I thought the "readline" command might do the trick, but can not seem to get it to read anything. I tried a little test program, I can not get it to read the second file. Can this be done in Perl? Your advise as to whether I am wasting my time would be greatly appreciated.

if (! open (InFile, "<$ENV{J_InputFile}"))
{
logMessage(INFORMATIONAL, 'Cannot open file: ' . $ENV{J_InputFile});
exit 2;
}

if (! open (InFile2, "<$ENV{J_InputFile2}"))
{
logMessage(INFORMATIONAL, 'Cannot open file: ' . $ENV{J_InputFile2});
exit 2;
}

if (! open (OutFile, ">$ENV{J_OutputFile}"))
{
logMessage(INFORMATIONAL, 'Cannot open file: ' . $ENV{J_OutputFile});
exit 2;
}

my ($File1Key, $File2Key);
my $File2Processed = "Y";
my $File2Line;

while (my $InFileLine = <InFile>)
{
chomp $InFileLine; #strip the carriage returns

$File1Key = substr($InFileLine, 0, 2);
logMessage(INFORMATIONAL, 'File1Key=' . $File1Key);

if ($File2Processed eq "Y")
{
$File2Line = (readline<InFile2>);
$File2Key = substr($File2Line, 0, 2);
logMessage(INFORMATIONAL, 'File2Key=' . $File2Key);
$File2Processed = "N";
}

if ($File1Key eq $File2Key)
{
$DataOutputRecord = substr($InFileLine, 0, 3) . substr($InFileLine, 4, 6);
print OutFile "$DataOutputRecord" . "\n";
$File2Processed = "Y";
}
else
{
$DataOutputRecord = $InFileLine;
print OutFile "$DataOutputRecord" . "\n";
}

}

if (! close (InFile))
{
logMessage(INFORMATIONAL, 'Cannot close file: ' . $ENV{J_InputFile});
exit 2;
}

if (! close (InFile2))

 

[KevinADC]

you don't need to use readline() for that, just assign the value of <> to a scalar:

$File2Line = <InFile2>;






- Kevin, perl coder unexceptional!

 

[gsdcrazy]

KevinADC,

So simple when you know what to do. Thanks. I don't think I would have ever tried that. I am sure I missed it in the books I have, but really needed to move ahead with this.

Thanks again,
GSDCrazy

 

[KevinADC]

You're welcome.

- Kevin, perl coder unexceptional!