AIX/Oracle tuning WIO problem

[RayMan]

I am running Oracle 8.1.5 on AIX 3 4 000942214C00. My sar report is as follows:

07:50:51 %usr %sys %wio %idle
07:50:52 4 1 1 94
07:50:53 3 0 0 96
07:50:54 2 2 1 96
07:50:55 16 3 1 80
07:50:56 20 2 1 78
07:50:57 2 0 0 98
07:50:58 6 0 1 93
07:50:59 0 0 3 97
07:51:00 0 0 2 98

I have noticed that I get wio nearly all the time reqardless of the load on the system. We run RAid-5 and I am having problems determining which process is accounting for the WIO. Is there a way to isolate WIO to a specific process id? Any ideas will be appreciated.

RayMan

 

[PSD]

Rayman,

The I/O column i.e. the third column is very low - the page orientation is out so that the idle column looks like it is under the WIO header.!!!!!! i.e. the fourth column is %idle!!!

Cheers

PSD
HAMCP Specialist

 

[RayMan]

Yes, I know, When I copied this from unix, the columns were skewed during the copy. The problem is that I get WIO even when the sys and user cpu consumption is very low, in some cases 0. Look at the last two times. WIO of 3 and 2 when sys & usr cpu is zero. As the load increases so do the WIO numbers. When I catch the system under load, the WIO exculates on the same scale shown at low utilization. The oracle db is about 2GB, and I have a 2GB buffer for oracle.
The machine has a total of 4GB of memory.

RayMan

 

[shiraz]

hi
Run iostat command against your disks...and watch
that whether your i/o load is distributed across the
disks or not.
As far as memory leakage or utilization by processes
is concerned AIX performance toolbox is best to be used.
You can use svmon,tprof and some other similar commands
to troubleshoot the matter.
To start with available resources,it is best to use
iostat,vmstat and even ps command to pin out the root
cause of the problem.

Regds
Shiraz To Every Problem There is a solution...........

 

[PSD]

Rayman,

Lets just clarify what WIO actually means - the figure is based on the highest CPU in terms of WIO it is not an average of all CPU's therefore it can be misleading. It also means that the CPU is idle waiting on an I/O activity to complete.

The lack of entry in the system and user columns could be a result of something accessing I/O directly - as it is only 1-2 percent it is not somthing to worry about. When it starts getting to 20-30% + you have an I/O bottleneck.

Look at the map page on filemon - this is used to identify I/O hotspots. It is a very useful AIX tool in determining the cause of I/O problems.

HTH

PSD
HAMCP Specialist

 

[RayMan]

My disks are logical volumes with the data stripped across multiple physical volumes. So an iostat tells me hdisk5 is busy. This does me little good this is not the activity for
a physical volume but a logical one. Is would be easier if
I could tell the pid causing the WIO.

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.0 603.5 24.7 6.0 58.9 10.5

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk0 14.0 57.9 14.0 0 58
hdisk1 16.0 57.9 14.0 0 58
hdisk3 0.0 0.0 0.0 0 0
hdisk4 0.0 0.0 0.0 0 0
hdisk2 10.0 83.8 18.0 0 84
cd0 0.0 0.0 0.0 0 0


Thanks
RayMan

 

[slgordon]

Try this...

ps vg |tail +2l |sort +4n -r |more

This should list the processes by PGIN which is...

"The number of disk I/Os resulting from references by the process to pages not loaded in core." Says the Manager to the person that invented the modem: "Tell me again why you need two prototypes?!?"

 

[tamuzi]

Hey,
I had exactly the same problem.
I calles my local supporter in IBM, and it found out that this is a known bug with this ersion of OS, which sometimes shows WIO in high percents although nothing is happening at all.
I got some PTFs which fixed my problem.

Try checking it.

 

[RayMan]

Thanks for all your input.

RayMan