Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations IamaSherpa on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Perl and 2GB files 1

Status
Not open for further replies.

louvail

Technical User
Dec 13, 2002
2
US
Is there a version of perl that handles files greater than 2GB? Our current version doesn't seem to (5.00404). I've had problems with readdir and rename so far.
 
if you try
open(IN,"$file") or die "cannot open $file\n";
while(<IN>){.....}

you are readding one line at a time

If on the other hand you do
my @lines=<IN>;
foreach $line(@lines){
...
}
you will need a lot of RAM to store everything in an array. The first way should work

svar
 
Perl 5.6 and higher support large files. Here's an excerpt from the perl56delta documentation file.

===========
Large file support

If you have filesystems that support ``large files'' (files larger than 2 gigabytes), you may now also be able to create and access them from Perl.

NOTE: The default action is to enable large file support, if
available on the platform.

If the large file support is on, and you have a Fcntl constant O_LARGEFILE, the O_LARGEFILE is automatically added to the flags of sysopen().

Beware that unless your filesystem also supports ``sparse files'' seeking to umpteen petabytes may be inadvisable.

Note that in addition to requiring a proper file system to do large files you may also need to adjust your per-process (or your per-system, or per-process-group, or per-user-group) maximum filesize limits before running Perl scripts that try to handle large files, especially if you intend to write such files.

Finally, in addition to your process/process group maximum filesize limits, you may have quota limits on your filesystems that stop you (your user id or your user group id) from using large files.

Adjusting your process/user/group/file system/operating system limits is outside the scope of Perl core language. For process limits, you may try increasing the limits using your shell's limits/limit/ulimit command before running Perl. The BSD::Resource extension (not included with the standard Perl distribution) may also be of use, it offers the getrlimit/setrlimit interface that can be used to adjust process resource usage limits, including the maximum filesize limit.
===============
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top