What is going on with available memory ?

[kletskov]

Hi all.
I'm in process of porting code from Windows to Sun solaris and we faced the problem of execution time.... Program works very slow on SUN workstation comparing to PC. I investigated runtime internals of the program and found out that the amount of available physical memory becomes very low (~ 7MB out of 512MB) -- I use sysconf to determine this amount. (On PC we never needed more than 100 -150MB RAM)
I think that system begins using swap, and I would like to avoid using it...
The questions are :

1. Is there any way to determine the size of memory that can be allocated (malloc) at some time without using a hard drive (swap) ?

2. Is there a utility that shows current using of swap by a process ?

3. Can I configure system/program/process to notify me in case of using swap (or not to use swap at all and get error "not enough memory" or something) ?

I'm new to unix and could not find a good documentation on solaris so far..

Thank you,
Pavel.

 

[marrow]

Commands to monitor swap as follows: -
swap -l
swap -s

ps -efly (not sure of the exact option) but the "SZ" column will give total size of each process in virtual memory

To Add / Delete swap space after creating new swapfile via mkfile command
swap -a /fullpath/to/swapfile will add
swap -d swapfile will delete

To seriously monitor your system
vmstat 2
runs every 2 secs and monitors free and swap memory
check "w" under procs, if > 5 these are swapped processes, high number of swaps means shortage of physical memory
r = run queue indicates jobs queuing - if > 5 this means insufficient CPU power in the system
b = processes blocked waiting for disk, high block rate indicates need for more memor

sar utility is also useful
sar -r 2 10 will monitor free memory and swap, 10 times at 2 sec intervals

see man pages for the above commands

Although I cannot be more specific, some of the above maybe of use.

 

[kletskov]

Thank you very much for the info. Problem seems to be not with swap. Example :
Before program1 starts working, there is 400MB free RAM (C function sysconf() gives this information as well as vmstat), then program1 works, no swap used according to vmstat and free RAM becomes 8MB. And it stays 8MB till the end of program work and AFTER program1 normally exited. Program1 uses malloc() to get memory.
I created program2 which asks system for 300MB of memory (tried malloc() and calloc() ) and fills all that space with some garbage (to make sure memory was actually allocated and check if swap was used).
The results are :
1. program2 GETS 300MB of RAM from the system (without using swap files) at the time of runnig program1 (free RAM shows 8MB at this point), or after program1 exited.
2. after program2 exits (if calls free() to free allocated memory) , free RAM becomes something like 280MB.

That is where the problem is. Inside program1 there is a check for free memory , which uses sysconf(). When it receives 8MB as the amount of free mem, it says program1 to use no more than 8MB (and program1 wants up to 150MB sometimes), that is why it works very slow. But, as shown above, the "really free" memory that could be allocated is not what vmstat/sysconf() returns.....

QUESTION : Does anyone know any way to get the size of memory that "ready to be allocated, or really free, or... you name it". Of course, one way I know is TRY to get the memory as I did in program2, then release it ... but it does not seem to be a good solution to me.
Can anyone explain what I described here ?

Thank you,
Pavel.

PS. Program1 worked for years under windows, there were no memory leaks. Does it sound like a memory leak to anyone ? But if it is , would I be able to reallocate leaked memory while program1 still runs ??

 

[marrow]

Hello Pavel,

My only advice on memory leakage is to monitor vmstat (or equivalent method). The "swap memory" column should not continually drop but should eventually recover some memory, if it doesn't this may indicate a memory leakage somewhere on the system. According to some old notes in my pocession: - "to determine which application has a memory leak look at the SZ column in the ps -el output" this may give you a clue.

Maybe somebody else out there has more to add.

Best of Luck

Marrow

 

[dobbyn]

If you are running Solaris 7, the free memory returned by vmstat (and probably sysconf) is not just the amount of memory used by applications, but also memory used by the system as file system cache. This is the way Solaris 7 improves i/o performance. That would explain why when your programs exit, the memory does not appear to be freed - the system is holding on to any file data it has cached.

My suggestion would be to just use malloc to allocate the memory and let Solaris handle the allocation. Malloc should return an error if there is not enough.