CPU wait???

[khalidaaa]

I've got this output from running the ps aux command

[edms]{root}/>ps aux | more
USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMAND
root 516 33.6 0.0 12 12 - A Dec 05 76390:04 wait
root 774 16.6 0.0 12 12 - A Dec 05 37745:21 wait
oracle8i 56852 3.7 4.0 36712 19332 - A 08:20:43 4:14 oracleENGP (LOCAL
oracle8i 42036 1.9 3.0 33312 15976 - A 08:19:44 2:14 oracleENGP (LOCAL
oracle8i 48126 1.4 2.0 28356 10492 - A 08:05:01 2:01 oracleENGT (LOCAL
root 32054 0.8 0.0 1796 1332 - A 07:40:41 1:34 smbd
root 54444 0.5 0.0 1812 1152 - A 11:52:50 11:58 smbd
root 53808 0.4 0.0 1716 1488 - A 07:22:45 1:02 smbd
oracle8i 12702 0.4 2.0 28676 11776 - A 22:12:40 5:48 oracleEDM3 (LOCAL
root 53380 0.4 0.0 1572 1576 - A 09:13:32 0:02 smbd
dmadmin 55258 0.3 1.0 12260 4120 - A 22:12:36 3:53 /usr/cimage/mtserroot 1290 0.2 0.0 12 12 - A Dec 05 521:54 lrud
oracle8i 36758 0.2 2.0 28496 11676 - A 07:06:32 0:34 oracleEDM3 (LOCAL

I'm just wondering what is this wait process that is running and taking 33.6% of the CPU???

I've read that it could be a zombie process or it could be that the CPU is idle!!! but this is not changing for 2 hrs and i'm facing slowness in the system!!! any suggestions??

Thanks

regards
Khalid

 

[p5wizard]

Those "wait" processes just mean that there are at least 2 CPUs waiting around for work to do - the "wait" processes add up all the idle time of your system.

The %idle in vmstat should roughly agree with the average %CPU of all your wait processes - there's 1 wait process for each CPU (or SMT thread) in the system.

If your system is slow, it wont be due to lack of CPU... Perhaps iostat can give you some clues (hotspot disk?)

HTH,

p5wizard

 

[khalidaaa]

That's the output of the iostat

[edms]{root}/>iostat 10

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.0 8.5 17.8 32.1 44.9 5.1

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk1 2.1 66.5 4.2 404615500 48689988
hdisk2 0.5 23.4 1.2 133749524 25864884
hdisk0 2.1 15.1 3.2 61362127 41556996
hdisk3 0.2 2.9 0.4 18184969 1873087
hdisk5 0.6 8.0 0.9 24700732 29658220
hdisk4 1.2 46.8 2.8 297548296 21392180
dac0 0.0 150.0 10.4 782700991 239416536
hdisk6 1.2 150.0 10.4 782700991 239416536
cd0 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.0 80.1 30.1 7.0 55.9 7.0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk1 11.2 60.0 14.6 560 40
hdisk2 0.6 2.4 0.6 0 24
hdisk0 7.9 57.6 9.9 208 368
hdisk3 0.1 0.8 0.2 8 0
hdisk5 0.6 2.4 0.6 0 24
hdisk4 0.5 2.4 0.6 0 24
dac0 0.0 0.0 0.0 0 0
hdisk6 0.0 0.0 0.0 0 0
cd0 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.0 79.4 11.8 6.1 78.2 3.8

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk1 3.7 30.8 4.1 36 272
hdisk2 2.5 19.6 3.3 16 180
hdisk0 2.6 14.8 3.3 44 104
hdisk3 1.7 8.3 2.1 32 51
hdisk5 1.0 4.4 1.1 8 36
hdisk4 1.2 4.0 1.0 8 32
dac0 0.0 4.8 1.2 24 24
hdisk6 0.1 4.8 1.2 24 24
cd0 0.0 0.0 0.0 0 0

 

[p5wizard]

Well, there's not a whole heap wrong with your system...

about 50% idle, not a lot of IO wait - I'm guessing that your Oracle instances are not ideally tuned to your system's capacity. Am I correct in deducing that you have multiple Oracle instances running on the same server?

Can you paste a topas screen? run "topas 5" for a few cycles, then type q, and copy/paste the output screen.


HTH,

p5wizard

 

[khalidaaa]

yes its correct that we have 4 oracle instances in there

and that's the topas output

Topas Monitor for host: edms EVENTS/QUEUES FILE/TTY
Thu Mar 2 12:13:43 2006 Interval: 2 Cswitch 518 Readch 658 12512 11.0 | Syscall 79 Writech 16545 0rnel |0 0.5 | | Reads Rawin 40.2 | ## | 1.7 |# | Wri 1s 1 Ttyo0t 800.2 | 0.5#############7.5## | Fork0 # Ige18 0.50 8.0 | 0 #### #| Execs 1 Na 01 0140.0 Runqueue Dirblk 02
en1work K0.5 I-Pac2 O-Pa015 0.0 4 1.0 4 0eue
lo0 0.5 2 1 0.0 0 1.0 0
0 0 PAGING 74 MEMORY 5 1 32.8hdisk0 19.4 120.0 27 239.9 00 Real,MB 8hdisk4 0.5 68.9h2isk4 100KB-Rea2.0B-W4it 1 0.0 00 % Comp 1.0hdi1 1.0.0 14 4.0 1 18 0.0 8 8.0PgspIn 029% Noncomp
1.0sk5 1.0 5 4.2 1 8 0.0 8.4 PgspOut 2 % Client
5 2 1.0 4 4 2 1 1 8 8 4 PageIn 1 20482ame 4 5 5 4 PID4CPU%1PgSp1Owner 8 8 PageOut 4 4 PAGING SPAC27.7topas 31912 1.3 0.9 root Sios 4 4 Size,MB 72.2topas 1912 1.0 0.9 root 0 % Used
topasver 31912 1.5 2.9 dmadmint NFS (calls/sec)0 % Free
mtserver 32920 0.3 3.8 dmadmin ServerV2 0
nfsd 17030 0.0 0.0 root ClientV2 0 Press:
oracle 44146 0.0 1.4 oracle8i ServerV3 "h" for help
aioserver 28896 0.0 0.0 root ClientV3 "q" to quit


i don't know if it make sense

Thanks p5wizard any way

 

[p5wizard]

I can't make sense of that ouput, please post it again, but inside a [ignore]

block.

example:

  your output goes here
       formatting and spacing
  stays       intact

[/ignore]

becomes:

  your output goes here
       formatting and spacing
  stays       intact

HTH,

p5wizard

 

[khalidaaa]

Topas Monitor for host:    edms                 EVENTS/QUEUES    FILE/TTY
Thu Mar  2 15:18:26 2006   Interval:  2         Cswitch     527  Readch        676            17175          1.0                Syscall      71  Writech   12401.0rnel    0.5   |         1                  |  Rea3s            Raw0n        52.7   |#   0     |         1                  |  Wri3es        1  Tty0ut       951.2  |####0.5   | #########.7####            |  For0s            Ig16s          7.2e      8.5   | 0                         #|  Execs            Namei         0.0                                              Runqueue         Dirblk
en1work  K0.5   I-Pac1  O-Pac1     0.0     1.0  Waitqueue
lo0       0.0        3       1     0.0     0.0
                                                PAGING       31  MEMORY      5130             41.4hdisk4    1.4      4.0       0     0.0      00 Real,MB       59.4hdisk0    1.4  KBPS6.0  TPS K1    12.0     0.0teals           0 Comp        0.5hdi4    1.0.4    4 4.0     1 0     0.0   8 8.0PgspIn          1% Noncomp
hdisk1    1.0      4.0       1     0.0     8.0  PgspOut       2  % Client
     5    0.5      2         0             4    PageIn        0             2048Name 2      5   PID2CPU% PgSp0Owner        4    PageOut       3  PAGING SPAC28.6topas         53872  1.0  0.9 root              Sios          3  Size,MB    71.3lrud           1290  0.3  0.0 root                             0 % Used
mtserver      32920  0.3  4.2 dmadmin           NFS (calls/sec)0 % Free
mtserver      51370  0.3  2.2 dmadmint          ServerV2       0
oracle        32920  0.0  4.0 oracle8           ClientV2       0   Press:
oracle        37424    0  1.2 oracle8i          ServerV3           "h" for help
lrud           129        0.0 root              ClientV3           "q" to quit

 

[RodKnowlton]

khalidaaa,

Could you please do the same (post ouput inside code tags) with the command "vmstat 5 10"? That is, if your system has performance problems.

To answer your original question, the cpu(s) always need to be doing something, so the "wait" processes take any cycles that no other process needs. For all but the most utilized machines, the "wait" processes will show up at the top of "ps aux" just about every time.

Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L
CompTIA Linux+
CompTIA Security+

 

[khalidaaa]

Thats the output for vmstat 5 10

but as you can see the system is almost idle coz its too early morning

kthr     memory             page              faults        cpu
----- ----------- ------------------------ ------------ -----------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa
 2  2 158882   128   0   5   3 184  515   0 353 16540 798 16 29 49  5
 0  0 158888   137   0   0   0   3    3   0 276  907 533  1  1 97  1
 1  0 158708   317   0   0   0   0    0   0 271  868 523  0  0 99  1
 0  0 158715   310   0   0   0   0    0   0 286 1160 574  2  1 96  1
 1  0 158715   310   0   0   0   0    0   0 268  843 509  0  0 99  1
 1  0 158895   239   0   7   0  37   43   0 286 1206 561  1  1 92  6
 0  0 158895   239   0   0   0   0    0   0 270  850 524  0  0 98  1
 0  0 158715   419   0   0   0   0    0   0 271  871 523  0  0 99  1
 1  0 158715   419   0   0   0   0    0   0 269  845 528  1  0 98  1
 1  0 158730   394   0   0   0   0    0   0 311 1181 626  2  1 95  2