NMON CPU_ALL % does not match sum of each CPU%

[gmarino]

I am having a problem reconciling the various CPU utilization numbers that nmon is reporting. On the SYS_SUMM and CPU_ALL reports, the CPU Utilization is much higher than if you average the CPU utilization from each individual CPU report (on the CPU_SUMM page.).

For example...here's some data from ONE nmon report...

CPU_ALL shows 40.4
vs
CPU_SUMM shows 26.2

WHY THE DIFFERENCE???

Here's the details

CPU_ALL shows :

User% 35.80283333
Sys% 4.6145
Wait% 1.931
Idle% 57.64983333
CPU% 40.41733333 <<*****


and CPU_ALL (which summarizes the data reported on the CPU_1, CPU_2...CPU_8 pages) shows:

CPU_SUMM User% Sys% Wait% Idle%
CPU01 25.3 4.3 3.4 67.0
CPU02 16.0 2.2 1.2 80.7
CPU03 26.5 4.5 3.4 65.5
CPU04 16.0 2.2 1.1 80.7
CPU05 31.6 4.4 3.4 60.6
CPU06 16.8 2.1 1.0 80.1
CPU07 17.5 5.4 3.8 73.3
CPU08 13.7 2.4 1.4 82.5
Average 20.4 3.4 2.3 73.8 26.2 <<<***



And more information (in anticipation of the questions to come)

lparstat -i
Partition Number : 1
Type : Dedicated-SMT
Mode : Capped
Entitled Capacity : 4.00
Partition Group-ID : 32769
Shared Pool ID : -
Online Virtual CPUs : 4
Maximum Virtual CPUs : 4
Minimum Virtual CPUs : 1
Online Memory : 19967 MB
Maximum Memory : 20480 MB
Minimum Memory : 256 MB
Variable Capacity Weight : -
Minimum Capacity : 1.00
Maximum Capacity : 4.00
Capacity Increment : 1.00
Maximum Physical CPUs in system : 4
Active Physical CPUs in system : 4
Active CPUs in Pool : -
Unallocated Capacity : -
Physical CPU Percentage : 100.00%
Unallocated Weight : -


Processor Speed 2096901000
Processor SMT enabled true
Processor SMT threads 2
Processor state enable
Processor type PowerPC_POWER5


Thanks in advance for your insights...

Greg

 

[khalidaaa]

IMHO maybe the CPU_ALL shows the output of all virtual CPUs (CPU1, CPU3, CPU5, CPU7) and CPU_SUMM account for all logical CPUs (CPU1 to CPU8)!

Can you test the above assumption on more than one snapshot of your system?

Regards,
Khalid

 

[gmarino]

I did some more digging - specifically on the IBM developerWorks nmon forum.

http://www.ibm.com/developerworks/forums/thread.jspa?messageID=13881140&#13881140

I think you are getting close to the answer. This is a 4 CPU server - 4 dual core CPU's with SMT turned on and set to 2. The odd numbered CPU's are the "real" or physical CPU's The even numbered CPU's are the "logical" CPU's - meaning they exist logically due to the use of Simultaneous Multi Threading (SMT). The logical CPU's are what I will call "bonus" CPU's. They don't ever work at 100% of the capacity of the physical CPU. According to the developerWorks forum post, they typically add 30-40% capacity.

I came up with a formula to calculate the "efficiency" of the logical CPU's - and I am posting this on the developerWork forum to get validation of it.

( Average of All Physical CPUs * 100% )
+ ( Average of All Logical CPUs * "Efficiency Level" )
= CPU_SUMM

Solve for the "Efficency Level" =
( CPU_SUMM - ( Average of All Physical CPUs * 100% ) )
/ ( Average of All Logical CPUs )