High IO wait

[jprabaker]

I'm getting a problem with high iowait on one of my servers. When running iostat I get results as follows:

tty: tin tout avg-cpu: % user % sys % idle % iowait
2.6 1053.3 2.8 1.3 47.2 48.8

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk0 0.0 0.0 0.0 0 0
hdisk2 0.2 1.6 0.4 8 0
hdisk3 0.0 0.0 0.0 0 0
hdisk1 0.0 0.0 0.0 0 0
hdisk4 0.0 0.0 0.0 0 0
hdisk5 0.0 3.2 0.4 0 16
hdisk6 0.0 3.2 0.4 0 16
hdisk7 0.0 0.0 0.0 0 0
hdisk8 0.0 0.0 0.0 0 0
hdisk9 0.0 0.0 0.0 0 0
hdisk10 0.0 0.0 0.0 0 0
hdisk11 0.0 27.2 5.0 56 80
hdisk12 0.0 26.4 5.2 56 76
hdisk13 0.0 4.0 1.0 0 20
hdisk14 0.0 0.0 0.0 0 0

There is nearly a 50% iowait yet none of the disks are busy. I've ran filemon but it doesn't really tell me anything useful.

The system runs AIX 5100-06.

Not sure were to start looking so any suggestions would be great.

Thanks,
John

 

[mrn]

What app is running?

Mike

"A foolproof method for sculpting an elephant: first, get a huge block of marble, then you chip away everything that doesn't look like an elephant.

 

[jprabaker]

SAP, oracle & patrol.

 

[mrn]

A little out of date but

http://users.ca.astound.net/baspence/AIXtip/download/oracle_aix_sap.htm

Mike

"A foolproof method for sculpting an elephant: first, get a huge block of marble, then you chip away everything that doesn't look like an elephant.

 

[mrn]

BTW

iostat doesn't just show disk info

% iowait Shows the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.

Get hold of a copy (If you haven't already) of nmon from

http://www-106.ibm.com/developerworks/eserver/articles/analyze_aix/

and start collecting data over a couple of days



Mike

"A foolproof method for sculpting an elephant: first, get a huge block of marble, then you chip away everything that doesn't look like an elephant.

 

[Yegolev]

recently we decided to bump up max aio servers, default is 10 which means oracle can do ten disk accesses at once. we went with max allowed, which is 1000. however our systems are rather large.

IBM Certified -- AIX 4.3 Obfuscation

 

[RMGBELGIUM]

Wait I/O can also mean network traffic, I suggest you check out your network traffic from and to the server during that high wait I/O


greetz

R.

 

[MoshiachNow]

Extra to RMGBELGIUM suggestion,run "topas" to see the whole picture .
For example if you would have a mismatch between FULL_DUplex/Half-Duplex/Autonegotiation setting between your host and the switch - you could get a very low network throughput,thus causing high wait with low disk usage (assuming that network is heavily used by your applications).

Long live king Moshiach !

http://www.7for70.com/