Facing performance issue on IBM AIX 5.3

[mrvivek82]

Hello Specialist,

We are facing performance issue on IBM AIX box.

Could anybody tell us something about following vmstat output.

[sadmpoc@cob1a012](/home/sadmpoc)$ vmstat -I 2

System configuration: lcpu=8 mem=5120MB ent=1,00

kthr memory page faults cpu
-------- ----------- ------------------------ ------------ -----------------------
r b p avm fre fi fo pi po fr sr in sy cs us sy id wa pc ec
2 32 0 2834632 15154 1132 0 396 366 854 2709 1754 34595 7757 44 28 23 5 0,80 79,7
7 28 0 2835789 37225 982 375 474 243 0 0 1857 50967 7963 44 24 28 3 1,48 147,8
4 30 0 2836894 34291 1383 0 467 0 0 0 1292 67865 5030 70 21 6 2 1,21 121,0
2 30 0 2838177 31885 3409 0 522 0 0 0 1854 45233 8724 72 21 4 3 1,32 131,9
9 28 0 2840613 26639 4072 25 500 0 0 0 1884 60061 8224 71 22 3 4 1,16 116,3
3 32 0 2841500 20939 2938 68 511 0 0 0 2087 24438 5885 73 19 6 2 1,09 109,2
4 30 0 2842070 15338 2419 23 510 0 0 0 2030 20912 3941 80 13 5 2 1,23 122,5
5 29 0 2842547 14788 3410 66 275 2045 2818 10439 2806 23456 7996 70 22 4 4 1,27 126,7
2 32 0 2842837 15162 2736 26 105 2476 2778 9630 2846 25566 3869 74 19 2 5 1,38 138,1
2 31 0 2842975 14484 2801 65 94 1930 2927 11977 2361 36663 3837 75 18 2 5 1,44 144,4
2 30 0 2843021 14818 3189 73 105 2165 2181 8517 2802 10867 5859 73 20 4 3 1,35 134,6
3 30 0 2843639 15345 3423 25 163 3485 3519 11427 3010 27264 4979 70 23 5 2 1,45 145,2
1 30 0 2843822 14214 3179 56 213 2025 2970 6572 2798 17880 7532 74 19 2 4 1,32 132,3
3 31 0 2843974 14674 2758 14 128 2680 2884 7532 2482 30957 3024 75 18 3 3 1,34 133,9
3 30 0 2844492 15679 2768 70 253 3179 3374 9864 2746 40298 5128 73 20 1 5 1,44 144,1
3 30 0 2844918 14908 2899 80 306 2435 3098 9715 2467 59968 5164 73 20 2 4 1,57 157,4
3 30 0 2845197 14843 3262 66 303 1949 2353 6560 2995 31506 8896 73 20 4 3 1,54 154,3
2 31 0 2846109 15245 3041 60 148 2409 3026 6462 2794 20767 7168 79 14 0 6 1,98 198,1
4 31 0 2846668 15214 3685 92 243 2673 3353 7830 3331 31134 12863 72 21 0 7 1,88 188,1
2 29 0 2847045 15064 3682 101 383 2464 3348 6027 3356 34301 14183 67 25 4 4 1,76 175,8

Memory on box = 5GB
Swap = 10GB

But currently swap space increasing rapidly, it is now 83.5%.

Kindly suggest.
Thanks in advance.

Vivek.

 

[Aixexplore]

Hello mrvivek82,

I suspect some virtual memory issue, It seems like it's paging in/out ,Also which Applications are running ? Need to perform some vm tunning for systems as well as within App. Possible post result of #svmon -P | grep -p Pid

Regards !

 

[mrvivek82]

Hello Aixexplore,

Thanks for you quick reply..

I am using Siebel application.


svmon -P | grep -p Pid

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1122426 siebsvc 2768841 7919 5602 76299 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
495672 siebmtsh 1489503 8010 10339 83828 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1147102 siebmtshmw 581816 7938 37757 210798 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
540878 siebmtshmw 553586 7947 37411 132270 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
503852 siebmtshmw 531681 7949 27805 111688 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1016034 siebmtsh 516341 7942 5879 77554 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
721012 siebmtshmw 495157 7932 34589 110086 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
716896 siebmtshmw 423757 7947 39536 111814 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
421950 siebmtshmw 406508 7947 38047 117735 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
663650 siebmtshmw 372529 7936 37021 135806 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
766086 siebmtshmw 341650 7947 37654 126744 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
622662 siebmtshmw 332460 7947 38628 116080 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1290404 siebmtshmw 325715 7949 3806 111485 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
802958 siebmtshmw 316359 7949 38631 111439 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
667732 siebmtshmw 297080 7956 40277 112260 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
430164 siebmtsh 290145 7924 77364 153625 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
794770 siebmtsh 274150 7926 10888 81558 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
815268 siebmtshmw 247340 7929 37787 110692 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
704620 siebmtshmw 247339 7929 37779 110694 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
712808 siebmtshmw 247294 7929 37779 110695 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
790662 siebmtshmw 246995 7949 37790 111172 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
983206 siebmtsh 227895 7933 4537 75393 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
643142 siebmtshmw 223998 7929 37810 110666 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
618560 siebmtshmw 223952 7929 37819 110683 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1380418 siebmtshmw 223004 7946 3806 109892 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
999496 siebmtsh 204664 7922 9477 80204 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
528468 siebmtshmw 203598 7946 25823 108943 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
675922 siebprocmw 195139 7924 29113 121122 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1175642 siebmtshmw 195020 7928 35456 106487 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1274076 siebsess 191861 7916 4319 85901 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
753796 siebprocmw 188246 7924 32347 118654 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
671822 siebprocmw 187796 7924 31131 117934 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
757888 siebmtshmw 187059 7946 111295 182178 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1179774 siebmtshmw 187054 7946 39040 109960 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
696454 siebmtshmw 187047 7946 39000 109915 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
729192 siebprocmw 185762 7924 34489 118991 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
884762 siebprocmw 185143 7922 31529 116536 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
413726 siebprocmw 184054 7924 32217 115970 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
749694 siebmtshmw 179315 7976 39852 110747 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
659540 siebmtshmw 179280 7946 38014 108871 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
397312 siebmtshmw 179277 7946 38034 108893 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
823448 siebmtshmw 179276 7946 39059 109971 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
970972 siebprocmw 177037 7924 31575 109987 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
651350 siebprocmw 175038 7924 33403 109983 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
389186 siebprocmw 174767 7924 33422 109982 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1036526 siebprocmw 174460 7924 33766 109983 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
770170 siebprocmw 174305 7924 33920 109984 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
897212 siebprocmw 174141 7924 33983 109983 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
966882 siebprocmw 173994 7924 34060 109984 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
688226 siebprocmw 173890 7924 35634 111968 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
741496 siebmtshmw 173260 7946 7218 78014 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
684122 siebprocmw 172658 7924 34714 109982 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1200188 siebsess 171168 7916 3806 80244 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
761992 siebproc 170962 7916 32390 113005 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
831652 siebprocmw 167841 7924 37224 109982 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
778376 siebprocmw 163841 7924 38932 109980 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
786574 siebproc 162974 7916 30053 104949 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
372922 siebproc 160796 7918 33473 106745 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
880820 siebproc 160639 7918 33603 106704 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
946384 siebproc 160321 7918 33841 106709 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
733302 siebproc 160288 7918 33847 106680 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
901308 siebproc 160220 7918 33921 106698 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
827556 siebproc 158815 7918 35257 106705 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
905406 siebproc 158623 7916 35434 106618 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
889018 siebproc 158612 7916 35350 106618 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
737346 siebproc 158605 7916 35348 106621 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
634984 siebproc 158602 7916 35378 106619 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
835748 siebproc 158576 7916 35407 106622 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
962788 siebproc 158575 7916 35401 106624 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
782486 siebproc 158575 7916 35396 106618 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
839852 siebproc 158568 7916 35404 106624 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
868536 siebproc 158549 7916 35405 106616 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
819354 siebproc 158545 7916 35412 106621 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
856238 siebproc 158544 7916 35404 106617 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
499788 siebproc 158543 7916 35456 106621 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
336016 siebproc 158542 7916 35404 106622 N N N

------------------------------------------------------------------------------- Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1097954 siebproc 158539 7916 35441 106620 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
958496 siebproc 158534 7916 35411 106619 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
630860 siebproc 158533 7916 35414 106620 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
655456 siebproc 157717 7916 8832 79641 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
950428 siebmtshmw 157703 7927 6019 76788 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1028100 siebmtshmw 157702 7927 6029 76796 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
934048 siebproc 157482 7916 4110 74637 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1220730 siebsess 157423 7916 8879 79465 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
708726 siebproc 150771 7916 35452 106616 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1224844 httpd 86044 8948 16077 97905 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
381154 java 85570 7984 19096 97727 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1327172 dbx 84784 7916 3806 75132 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
340056 siebsvc 72099 7987 8729 79610 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
204800 syslogd 72034 7916 3854 74295 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
163842 kmqagent 71439 7936 5403 76471 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1286206 dbx 71229 7916 3806 75092 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1253454 topas 71138 7916 5250 76399 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1241130 topas 71131 7916 5250 76392 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1032208 regss 71128 7944 5192 75757 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
196702 tlmagent 70956 7921 4495 75390 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
376878 IBM.CSMAgentR 70688 7932 4245 74849 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
454882 rmcd 70675 7924 4268 74872 N Y N
-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1323154 sshd 70654 7916 3806 74443 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1163286 sshd 70649 7916 3806 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1257704 sshd 70646 7916 3806 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
843840 sshd 70645 7916 3807 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1269916 sshd 70643 7916 3808 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
512070 IBM.ServiceRM 70636 7932 4088 74627 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1331372 bash 70617 7916 3806 74357 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
479266 bash 70615 7916 3806 74355 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1228982 bash 70612 7916 3806 74352 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1171476 httpd 70611 7916 3833 74307 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1376374 bash 70610 7916 3806 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
245962 kmcrca 70589 7934 4780 75435 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1126536 bash 70579 7916 3806 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1294400 bash 70579 7916 3806 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1282196 bash 70579 7916 3806 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1364168 bash 70579 7916 3806 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
700578 bash 70579 7916 3806 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
872568 bash 70577 7916 3806 74317 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1265792 bash 70566 7916 3860 74356 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1138806 bash 70564 7916 3902 74381 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1216690 bash 70560 7916 3905 74361 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
647298 httpd 70547 7916 3874 74289 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
921600 bash 70547 7916 3899 74354 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
745658 bash 70544 7916 3896 74355 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
987218 bash 70543 7916 3924 74366 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
327812 xmwlm 70543 7916 5359 75965 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
450722 tecad_logfile 70539 7920 3985 74491 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
507918 getty 70534 7916 3886 74316 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1249330 ksh 70533 7916 3806 74352 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1306632 ksh 70533 7916 3806 74352 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
852124 ksh 70533 7916 3806 74352 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1167598 ksh 70531 7916 3806 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1298564 ksh 70523 7916 3806 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1233150 ksh 70522 7916 3806 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1343672 ksh 70522 7916 3806 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1339604 ksh 70522 7916 3806 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1360070 ksh 70522 7916 3806 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1134682 sshd 70506 7916 3915 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1237098 sshd 70504 7916 3922 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1081510 sshd 70502 7916 3951 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1056964 sshd 70502 7916 3951 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
393430 lcfd 70502 7919 3929 74433 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1192048 sshd 70499 7916 3921 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
581828 sshd 70499 7916 3952 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
774312 sshd 70499 7916 3950 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
930024 sshd 70498 7916 3948 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
938166 sshd 70497 7916 3953 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
532658 sshd 70497 7916 3952 74436 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
942282 sshd 70496 7916 3947 74432 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
344104 sshd 70495 7916 3939 74425 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
692354 sshd 70495 7916 3945 74426 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
352442 sshd 70491 7916 3850 74323 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
475162 IBM.HostRMd 70486 7925 4056 74582 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1142818 bash 70477 7916 3925 74358 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1069238 bash 70477 7916 3891 74320 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1159278 bash 70477 7916 3892 74321 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1208476 bash 70477 7916 3890 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
811018 bash 70477 7916 3888 74318 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
876664 bash 70477 7916 3892 74319 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1019916 bash 70477 7916 3936 74365 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
188438 muxatmd 70463 7916 3841 74369 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
143528 syncd 70451 7930 3894 74381 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
258204 aixmibd 70450 7916 3909 74398 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1183962 ksh 70443 7916 3898 74339 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1155082 rootsh 70438 7916 3806 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1277956 rootsh 70438 7916 3806 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1335450 rootsh 70438 7916 3806 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1347814 rootsh 70438 7916 3806 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1093862 ksh 70435 7916 3917 74345 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1011714 mwrpcss 70432 7923 4211 74756 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1089766 sshd 70427 7916 3965 74435 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
237814 cron 70421 7916 3950 74408 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
438494 xntpd 70421 7916 3856 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1077348 ksh 70402 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1187956 ksh 70402 7916 3919 74339 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
536706 ksh 70402 7916 3919 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1118422 ksh 70401 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
589852 ksh 70401 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
516232 ksh 70401 7916 3919 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1003594 ksh 70399 7916 3919 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1048630 ksh 70399 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
917706 ksh 70399 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1085562 ksh 70399 7916 3925 74344 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
610410 ksh 70399 7916 3918 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
626804 ksh 70399 7916 3920 74338 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
909456 ksh 70398 7916 3933 74351 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1105998 ksh 70398 7916 3931 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1052832 ksh 70398 7916 3931 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1130672 ksh 70398 7916 3931 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
221270 ksh 70398 7916 3932 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
442442 ksh 70398 7916 3931 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
569352 ksh 70398 7916 3930 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
606346 ksh 70398 7916 3932 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
864378 ksh 70398 7916 3930 74350 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
483440 grep 70391 7916 3806 74260 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
725180 srvrmgr 70390 7916 5361 75858 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
106654 errdemon 70389 7916 3918 74345 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1 init 70387 7916 3913 74361 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
319728 iostat 70386 7916 3833 74278 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1196172 rootsh 70384 7916 3846 74307 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
860360 rootsh 70380 7916 3868 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
954406 rootsh 70380 7916 3848 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1060998 rootsh 70377 7916 3869 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1204394 rootsh 70377 7916 3871 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
807068 rootsh 70377 7916 3868 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
639086 rootsh 70376 7916 3866 74302 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
262302 automountd 70373 7918 4119 74557 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
593968 IBM.DRMd 70359 7934 4118 74630 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
356384 qdaemon 70357 7916 3835 74264 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
491770 IBM.ERrmd 70353 7929 4210 74708 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
401658 IBM.LPRMd 70351 7927 4112 74619 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
848092 rootsh 70349 7916 3876 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
991342 rootsh 70349 7916 3875 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
995538 rootsh 70349 7916 3876 74305 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
213038 IBM.AuditRMd 70349 7925 4136 74622 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
254092 rpc.mountd 70342 7919 4168 74597 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
270570 rpc.statd 70342 7919 4137 74572 N Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
225482 portmap 70340 7916 4018 74445 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1151034 sftp-server 70339 7916 3956 74385 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
979192 inner-watchdo 70339 7916 3859 74283 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
192604 biod 70339 7916 3833 74248 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
180392 srcmstr 70339 7916 3970 74403 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
209108 rpc.lockd 70339 7916 3865 74285 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
241864 dpid2 70339 7916 3916 74347 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
217314 nfsd 70339 7916 3838 74256 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
487432 dtlogin 70339 7916 3892 74331 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
286956 inetd 70339 7916 3881 74312 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
360668 uprintfd 70339 7916 3821 74233 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
364786 writesrv 70339 7916 3862 74280 N N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
548878 amqzlaa0_nd 21955 7990 17543 37654 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1044524 amqzlaa0_nd 21544 7967 11467 31197 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
573472 amqzmuc0 21456 7970 11331 30999 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
176214 shlap64 20737 7961 5702 26402 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
598142 amqzxma0_nd 20724 7967 10277 29962 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
577568 amqzfuma 20548 7965 9721 29326 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
311468 kuxagent 20483 7980 6954 27486 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
458976 amqzlaa0_nd 20141 7972 9089 28906 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
315616 amqzxma0_nd 20115 7967 9293 29024 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
291038 amqzmuc0 19986 7970 9139 28814 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
385276 amqzfuma 19967 7965 8998 28679 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
168080 stat_daemon 19738 7961 5772 25455 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
602234 amqrmppa 19645 7969 15743 35197 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
565270 amqzmur0 19635 7967 7387 26849 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
561174 amqrrmfa 19559 7965 7361 26750 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
557076 amqzdmaa 19509 7965 7265 26671 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
544780 runmqchi 19502 7966 7479 26851 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
1114276 svmon_back.64 19450 7961 5635 25063 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
307426 kuma610 19442 7991 7226 26837 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
524296 amqpcsea 19436 7964 7106 26496 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
409800 amqzmur0 19407 7967 6515 25837 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
463074 runmqchi 19389 7966 6530 25804 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
467172 amqpcsea 19373 7964 6181 25507 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
405704 amqrrmfa 19366 7965 6433 25728 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
331800 amqzdmaa 19344 7965 6352 25664 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
418002 amqzmgr0 19293 7965 6031 25277 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
233720 snmpd64v1 19265 7961 5804 25067 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
323748 kux_vmstat 19241 7961 5740 24977 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
295074 hostmibd64 19231 7961 5722 24958 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
249878 nfs_stat 19230 7961 5669 24886 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
552976 amqzmgr0 19224 7965 6037 25277 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
471270 runmqlsr_nd 19211 7965 6056 25269 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
299160 ifstat 19193 7961 5744 24943 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
434322 runmqlsr_nd 19179 7965 6055 25269 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
303142 runmqsc 19175 7961 6010 25234 Y N N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
127046 j2pg 9391 8010 4047 13109 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
200868 nfsd 8962 7997 3727 12664 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
139374 kbiod 8942 7992 3743 12659 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
45078 pilegc 8890 7989 3855 12592 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
131148 dog 8806 8113 3775 12556 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
282860 rpc.lockd 8793 7988 3727 12495 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
24588 psmd 8740 7983 3775 12442 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
16392 lrud 8708 7983 3759 12442 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
86058 gil 8706 7981 3743 12424 Y Y N

-------------------------------------------------------------------------------
Pid Command Inuse

 

[hfaix]

Is this new behavior or a new implementation?

You have a lot of non-standard Siebel stuff in your svmon. What are all the ssh, bash, ksh, and rootsh processes?

You can list top five memory consumers with
svmon -Put +5

You can list what's in paging with
svmon -Pg

If everything is valid, you need more memory.

I see you have 8 logical processors...is this 4 core with SMT turned on?

 

[MoreFeo]

In vmstat output I see a lot of blocked processes (b column). This is usually caused by processes waiting for IO.

You may take a look at this problem.

 

[cryptoadm]

I'd rather see 'vmstat -v' and 'vmstat -s' than svmon output.

Also need a 'pstat -A | grep aios | wc -l' to see how many aio servers are running.

Also check smitty aio to see how many aio servers are setup as well as priority and max requests.

 

[cryptoadm]

Run 'vmo -a' and 'ioo -a' too.

 

[mrvivek82]

Hello hfaix, cryptoadm, MoreFeo

Thanks for your reply.

Outputs are:

bash-2.05$ vmstat -s
94975198 total address trans. faults
17646469 page ins
22207070 page outs
2567814 paging space page ins
8600058 paging space page outs
0 total reclaims
29083711 zero filled pages faults
4503711 executable filled pages faults
70671380 pages examined by clock
383 revolutions of the clock hand
16062135 pages freed by the clock
2137124 backtracks
52024 free frame waits
0 extend XPT waits
13487882 pending I/O waits
22135460 start I/Os
20444788 iodones
189155889 cpu context switches
38034443 device interrupts
13007195 software interrupts
64247686 decrementer interrupts
80601 mpc-sent interrupts
80601 mpc-receive interrupts
2120803 phantom interrupts
0 traps
1098878513 syscalls

bash-2.05$ vmstat -v
1310720 memory pages
1232401 lruable pages
242032 free pages
6 memory pools
153628 pinned pages
80.0 maxpin percentage
20.0 minperm percentage
80.0 maxperm percentage
5.7 numperm percentage
71015 file pages
0.0 compressed percentage
0 compressed pages
5.7 numclient percentage
80.0 maxclient percentage
71015 client pages
0 remote pageouts scheduled
16939 pending disk I/Os blocked with no pbuf
4245691 paging space I/Os blocked with no psbuf
2484 filesystem I/Os blocked with no fsbuf
0 client filesystem I/Os blocked with no fsbuf
2534 external pager filesystem I/Os blocked with no fsbuf
0 Virtualized Partition Memory Page Faults
0.00 Time resolving virtualized partition memory page faults

Siebel Server, Getway Server, Web server is running on the same machine.

I am not able to run following commands because of restricted access on the server.Sudo access is also not possible.

'pstat -A | grep aios | wc -l'
vmo -a , ioo -a

And I think output from svmon -Put +5 having valid processes.

Please kindly suggest because still paging swap is of 94.5%.

Thanks in advance.
Vivek.

 

[KenCunningham]

Vivek. If people are expecting you to troubleshoot this issue/these issues, you really have to make it clear that you need the tools, including the permissions to do so!

Some days are diamonds, some days are rocks - make sure most are the former.

 

[cryptoadm]

4245691 paging space I/Os blocked with no psbuf

You need to find why it's paging. I would assume 5G for a database is enough and the web server if it's purley httpd and not an app server. What kind of disk are you using? All internal or do you have external SAN like EMC or IBM?

You need to see your vmo and ioo settings and make sure lru_file_repage=0 and also I would suggest you pin memory. You can set maxperm% at 90 and minperm% at 5% or even 3% though with 5G, 10% would probably be better although your numperm is at 5.7 so I would use 5%.

Is SMT active? Check with smtctl. You can also set your AIXTHREAD_SCOPE=S in /etc/environment.

You also need to check your AIO servers because if they are at the default that can kill you.

 

[khalidaaa]

Although looking at your minperm and maxperm which indicate that you might never had any memory tuning but there are some time that it is just better to add more memory than to worry about tuning if the memory bottleneck continues to happen. Here are some links that i use to identify and tune memory problems:

http://www.tek-tips.com/viewthread.cfm?qid=1446162&page=1

http://www.tek-tips.com/viewthread.cfm?qid=1302007&page=3

Regards,
Khalid