Bottleneck determination from VMSTAT command

[pwcsa]

Hi all,
Can anyone tell me what your thoughts are from the output of the following vmstat command? The idle CPU is very low & the fre memory hits the minimum (120). My thought is lack of memory...

kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa

0 6 555091 26343 0 2 0 0 0 0 1316 1040 824 0 1 23 76
0 7 555093 24540 0 12 0 0 0 0 1261 1048 824 2 2 23 73
0 7 555052 21350 0 11 0 0 0 0 1264 1142 1132 3 3 25 68
0 10 555101 10782 0 12 0 0 0 0 1536 1351 1899 5 4 14 77
0 7 555104 8504 0 3 0 0 0 0 1301 1023 798 0 2 20 78
0 8 555106 153 0 12 7 297 1620 0 1477 1283 1469 4 4 15 77
0 8 555109 120 0 7 36 928 2466 0 1490 1210 1788 3 3 18 76
0 15 555112 170 0 19 8 1553 4376 0 1813 1909 4395 2 7 17 74
0 11 555117 123 0 14 38 1373 4216 0 1735 1875 3987 2 6 18 73

 

[RudiVVC]

I think that, too.
As you see, heavy paging starts, when the
amount of free memory is going down to 153.

No memory -> machine has to swap.

Perhaps you should have a look, which
processes use the memory by typing

svmon -Pt 5

You'll get the 5 most memory consuming jobs.

 

[Morsing]

I don't see any severe paging happening.

The page stealer starts to reclaim pages but that doesn't mean you're memory constrained.
It looks strange to me that there are no running processes but more than ten blocked processes and the wait column is 75%.

Would processes claiming pages show up in the blocked column??

Cheers

Henrik Morsing
Certified AIX 4.3 Systems Administration
& p690 Technical Support

 

[mrn]

what app are you running?

--
| Mike Nixon
| Unix Admin
|
----------------------------

 

[pwcsa]

There was 3 oracle instances running and Cayenta batch queue process running as well. It was strange to me for the wait column reached to 75%. Is it indicating I/O bottleneck as well?

 

[dblair]

Correct, What is the disk config ?

 

[AIXSPadmin]

the b under kthr is and indicator of threads waiting on disk i/o.

The po is not near zero, and the fr:sr is at or over 1:4 which both signal memory problems.

So, you have both disk i/o AND memory contention going on with your system.

 

[pwcsa]

The file system layout was designed incorrectly. The fs are sitting on a single disk and mirrored it across. I will re-implement the fs layout and add additional memory. Thank you all so much for your input and time.

Best Regards.