clearing memory used by hash 1

[Aneusomy]

Hi all,
Does anyone know how to empty a hash and reclaim the memory, without exiting the script?
I've got a script that loads a 390meg gene list into memory, then when I'm done with it I want to empty the hash so that I can reclaim the memory and move onto the next memory-intensive op. I've tried:
undef %hash;

and also:
%hash = ();

I found undef %hash actually empties the hash (trying to parse it shows it's empty), yet when I monitor the script with 'top' in *nix I see it is still claiming 390megs.
Any ideas?
TIA,
Ben

 

[stillflame]

I really don't know, but my instinct says that maybe perl has allocated the memory for itself, so the system still sees it as in use, while it actually isn't.
If, after the "undef %hash;" line, you added something else which would take up some visible chunk of memory, like just reading in a 1 meg file into an array or something, you could then compare the final 'top' readings from that program with the original. if the new one doesn't actually increase in memory usage, that could mean that the memory is free, but only inside of perl.
yeah, in fact, thinking back to my (miniscule) knowledge of C, the alloc() and free() combo sucks in memory with alloc, but only frees it for use by the C program running, not the system. if my idea does work, please let me know. "If you think you're too small to make a difference, try spending a night in a closed tent with a mosquito."

 

[Aneusomy]

Hi stillflame,

Well, found the answer to the question and thought I'd let others know too:
just as you speculated, the memory that gets 'eaten' by Perl remains within Perl (i.e. the O/S cannot reclaim it) and is free for reuse within Perl. This memory, however, is not released to the O/S until the script ends. This means, the longer your script runs (my scripts work with 100Mb's files, which sometimes have to load 100-1000's of unique keys into hashes), the more memory the script gobbles, and the less memory available to the O/S.
This kinda sucks. Hope there will be something to fix this in Perl 6.

See:

http://mail.gnome.org/archives/gtk-perl-list/2000-March/msg00043.html

http://www.rocketaware.com/perl/perlfaq3/How_can_I_free_an_array_or_hash_.htm

Ben

 

[tsdragon]

This is supposed to clear the hash, but I don't know if it will return the memory to the operating system or not:

undef %hash;

Tracy Dryden
tracy@bydisn.com

http://www.bydisn.com

Meddle not in the affairs of dragons,
For you are crunchy, and good with mustard.

 

[Haunter]

The other option is not reading the entire Hash into memory and using a database to buffer the load and used just the infomation you need out of the hash.

Are you performing tasks on the entire 390megs of the hash or just sections of it.

I am sure you probably thought of the idea but if you free up that much memory maybe you could run the other tasks simultanously.