Out of memory Error but having ENOUGH memory!

[fd]

Hi,

I have a strange memory problem on an H80 under AIX 4.3.3:

I generated a text file which is 140MB and I try to edit it using vi (and/or a self written perl hack). "monitor" shows
-----------CUT HERE---------------------
...
Runnable (Swap-in) processes 0.00 (1.00) load average: 0.00, 0.02, 0.04

Memory Real Virtual Paging (4kB) Process events File/TTY-IO
free 116 MB 974 MB 238.42pgfaults 157 pswitch 0 iget
procs 209 MB 49 MB 0.0 pgin 580 syscall 36 namei
files 185 MB 0.4 pgout 23 read 11 dirblk
total 512 MB 1024 MB 0.0 pgsin 4 write 320374readch
IO (kB/s) read write busy% 0.0 pgsout 2 fork 171625writech
...
-----------CUT HERE---------------------
and an lsps -a shows:
Page Space Physical Volume Volume Group Size %Used Active Auto Type
hd6 hdisk0 rootvg 1024MB 5 yes yes lv
-----------CUT HERE---------------------


Both vi and the perl script (which tries to read all the 140MB into a hash) terminate with "Out of memory" if the free mem counter of "monitor" degrades to 0 while reading the file. The machine has (as u can c above) 512MB physical RAM and 1GB paging space - so it should NOT be a problem for the system to read in 140MB!?

At 1st, I tried it with perl 5.6.0 which has been compiled with a cflag of "-qmaxmem=16384". So I thought this behaviour would result due to this limit (BTW: what does this flag mean?). But because even vi terminates
I am out of thoughts now... Can somebody explain that to me please?

Here is another output from vmstat 10 10 while executing (script terminates at line 3 of data output):
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
0 0 46403 29684 0 0 0 0 0 0 156 182 113 1 1 95 3
0 1 60456 15599 0 0 0 0 0 0 326 923 270 29 2 66 3
1 1 46438 32992 0 0 42 337 822 0 341 750 236 42 3 54 1
0 1 46402 33040 0 0 0 0 0 0 245 151 84 0 0 99 0
0 1 46409 33033 0 0 0 0 0 0 320 621 358 1 0 95 4
0 1 46412 33030 0 0 0 0 0 0 404 1434 661 1 1 88 10
0 1 46409 33033 0 0 0 0 0 0 255 317 107 0 1 99 0
0 1 46419 33024 0 0 0 0 0 0 321 691 352 0 1 95 4
0 1 46376 33075 0 0 0 0 0 0 602 1888 1351 4 2 80 15
0 1 46376 33074 0 0 0 0 0 0 250 172 81 0 0 99 0


and ulimit -a says (root user):
core file size (blocks) 1048575
data seg size (kbytes) 131072
file size (blocks) 1048575
max memory size (kbytes) 262144
open files 2000
pipe size (512 bytes) 64
stack size (kbytes) 32768
cpu time (seconds) unlimited
max user processes 262144
virtual memory (kbytes) unlimited

 

[fd]

Ok, I guess I got it: ulimit -a shows the limitations. I've changed data segment size and memory to:

root@fnet4:/fnsw/local/logs/elogs> ulimit -a
core file size (blocks) 1048575
data seg size (kbytes) 1048575
file size (blocks) 1048575
max memory size (kbytes) 1048575
open files 2000
pipe size (512 bytes) 64
stack size (kbytes) 32768
cpu time (seconds) unlimited
max user processes 262144
virtual memory (kbytes) unlimited


and the system starts using more paging space (but the perl program still terminates, now with "Illegal Instruction&quot. I will check that.....