Wait I/O % =100% 2

[admincff]

On two RS/6000 (bi-processors type 7013-J50), nmon command return a CPU usage of 100% juste for Wait%, as shown below :

CPU Utilisation +-------------------------------------------------+
CPU User% Sys% Wait% Idle|0 |25 |50 |75 |100|
0 1.0 0.5 98.5 0.0

http://WWWWWWWWWWWWWWWWWWWWW>

1 0.0 0.0 0.0 100.0| >
0.5 0.2 49.3 50.0|

http://WWWWWWWWWW>

The iostat command return no significiant information about Disk I/O.
And no particular activity is observed on this 2 hosts.
I've tried before to reboot one of this 2 hosts, but nothing has changed...

Could someone give me informations about this???
Is there a known problem observed on multi-processors hosts ???
Thanks.

 

[MoshiachNow]

Hi,

What does "topas" show?

This could hint you on some stuck process consuming all the CPU.

ALso,check for top 10 CPU usage by process:

ps aux | sort -r +2 |head -10

"Long live king Moshiach !"
h

http://www.noahide.com/movement.htm

 

[admincff]

the &quot;ps aux | sort -r +2 |head -10&quot; give a usage CPU < 10%.
For the topas command, it's not install on my host

I've read on IBM support site some informations on vmstat tool and some comments :



About vmstat tool :

NOTE: The wa column on SMP machines running AIX Version 4.3.2 or earlier is somewhat exaggerated. This is due to the method used in calculating wio .

What do you think about it ???

 

[MoshiachNow]

Hi,
Possibly it's an issue of the SW level update.

However,please post output of the following:

vmstat 2 10

Thanks &quot;Long live king Moshiach !&quot;
h

http://www.noahide.com/movement.htm

 

[admincff]

vmstat 2 10 result :

kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
0 1 48751 4582 0 0 0 74 175 0 210 407 69 19 28 25 28
0 3 48751 4459 0 0 0 0 0 0 553 132 36 33 3 15 50
0 3 48751 4335 0 0 0 0 0 0 597 127 39 35 2 14 50
1 3 48751 4205 0 0 0 0 0 0 562 110 34 35 2 14 50
0 3 48751 4084 0 0 0 0 0 0 578 110 42 32 7 16 44
0 3 48751 3964 0 0 0 0 0 0 562 122 35 34 1 14 50
0 3 48751 3842 0 0 0 0 0 0 490 122 36 34 2 14 50
0 3 48751 3731 0 0 0 0 0 0 470 126 40 31 2 23 44
1 3 48751 3603 0 0 0 0 0 0 531 130 42 33 2 49 16
1 3 48751 3483 0 0 0 0 0 0 551 127 38 35 1 50 14

 

[MoshiachNow]

In this posting the system is less then 50% busy ... &quot;Long live king Moshiach !&quot;
h

http://www.noahide.com/movement.htm

 

[admincff]

Yes, but WaitI/O % is full (=100%) on my first proc

 

[MoshiachNow]

Ok.

Then I would run diag both &quot;system verification&quot; and &quot;problem determination&quot; selecting all the CPUs.

Also - check errpt for anything funny.
&quot;Long live king Moshiach !&quot;
h

http://www.noahide.com/movement.htm

 

[Yegolev]

Could be some deconfigured processors, I forget how that works on a J50. If you have a 2-way machine running at 50%, then you lose a proc, you would be running at 100%, right? You will want to call IBM Hardware. IBM Certified -- AIX 4.3 Obfuscation

 

[admincff]

Thanks for your help..
but ...
Diags return every proc is OK on both hosts.
Nothing to deal with errpt. Everything is OK...

 

[MoshiachNow]

Possibly a bug with pmon on this sw level - since vmtune and topas show normal. &quot;Long live king Moshiach !&quot;
h

http://www.noahide.com/movement.htm

 

[t1mberwulf]

What I would suggest is to load nmon also as it seems to be the favoured tool at the moment for I/O tracking

 

[t1mberwulf]

Just to add where to find the tool and a brief explanation
Q: Where can I find nmon ?
A: The nmon tool is helpful in presenting all the important performance tuning information on one screen and dynamically updating it. The tool works on any dumb screen, telnet session, or even dial-up line. In addition, the tool is very efficient. It does not consume many CPU cycles, usually below 2%. On newer machines, CPU usage is well below 1%. It collects a lot more performance information than topas.
The nmon tool can also capture the same data to a text file for later analysis and graphing for reports (spreadsheet format .csv).
The nmon tool was developed to support benchmarks and performance tuning for IBM internal use but by popular demand is given away to deserving friends.


Download at

http://www-1.ibm.com/servers/esdd/articles/analyze_aix/index.html.

 

[admincff]

Thanks, but as you could see on my first request, nmon is the tool i'm using and with i'd observed the wait I/O % load

 

[Yegolev]

Perhaps it is simply due to nmon being a new app and the J50 being a microchannel box from around 1997 or 1998. The RS6K/SMP architecture could be fooling nmon, perhaps it only works reliably on CHRP? I would suggest relying on older, standard tools, especially if they all agree and nmon is the odd-man-out. If I had a microchannel box laying around somewhere that was not at 4.2.1 I might test this. IBM Certified -- AIX 4.3 Obfuscation