Hi,
We have a seriuos performance problem with our AIX-P-series server & AIX 4.3.3
The P-series is a S85 server with 8 CPU and 24 GB internal memory (and 6 GB Paging space)
We are using a Oracle 8.1.7 database and have around 1000 oracle connections.
The last month we are have more and more Paging to disk and have on the busy hours (10:30 + 15:30 CET) a 100 % CPU load
During to IBM this machine can handle arounf 2500 oracle connections....
So what is wrong with our settings?
Why do it already Page to disk?
For your info i will sent you some info from the machine:
#lsattr -E -l sys0 -a realmem
realmem 25165824 Amount of usable physical memory in Kbytes False
#lsps -s
Total Paging Space Percent Used
6144MB 24%
#lsps -a
Page Space Physical Volume Volume Group Size %Used Active Auto Type
paging01 hdisk1 pagevg 2048MB 24 yes yes lv
paging00 hdisk2 perfvg 2048MB 24 yes yes lv
hd6 hdisk0 rootvg 2048MB 24 yes yes lv
# vmstat 2
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 3 1470223 210508 0 0 0 1422 1282 0 414 1971 1500 88 94 99 59
3 7 1465485 214707 0 0 0 0 0 0 2848 31953 14591 30 18 26 27
3 8 1467528 209350 0 0 0 0 0 0 3217 40869 25871 27 22 23 28
3 8 1467812 204489 0 0 0 0 0 0 3082 28764 31617 24 22 26 28
2 7 1468104 198725 0 0 0 0 0 0 3011 18157 33753 19 14 30 36
4 7 1469511 192674 0 0 0 0 0 0 2993 30201 31152 19 26 28 27
3 7 1469538 190357 0 0 0 0 0 0 2162 20648 18488 13 21 32 34
2 6 1469998 189132 0 0 0 0 0 0 1979 21613 13117 14 19 39 29
3 5 1470587 187379 0 0 0 0 0 0 1947 23281 12188 15 18 36 31
2 6 1472608 184982 0 0 0 0 0 0 2734 51722 12550 21 22 38 18
5 7 1467856 188993 0 0 0 0 0 0 3088 78239 13191 20 35 23 21
3 6 1473987 176793 0 0 0 0 0 0 2783 25293 29238 20 36 22 23
3 5 1468310 176252 0 0 0 0 0 0 2961 28108 34162 19 34 23 24
6 7 1465294 174925 0 0 0 0 0 0 2946 38061 31417 18 34 24 24
4 5 1465111 173676 0 0 0 0 0 0 2368 28924 13782 29 30 18 23
5 6 1463706 173446 0 0 0 0 0 0 2135 25798 14247 29 28 23 21
6 5 1462929 175498 0 0 0 0 0 0 1944 20058 10553 28 21 29 22
6 5 1463532 188869 0 0 0 0 0 0 1731 23732 10750 24 24 32 21
7 6 1463086 185838 0 0 0 0 0 0 2570 19635 28689 17 32 31 21
3 6 1462752 182424 0 0 0 0 0 0 2605 17292 35752 15 31 28 27
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 10 1462565 177337 0 0 0 0 0 0 2867 11434 36568 11 19 33 36
3 5 1462554 172417 0 0 0 0 0 0 2590 17417 31772 13 14 38 36
6 8 1463786 166770 0 0 0 0 0 0 2516 32620 28999 17 22 36 25
1 4 1464677 161805 0 0 0 0 0 0 2659 24602 26877 17 12 45 25
2 7 1463744 156665 0 0 0 0 0 0 2523 31844 32554 16 15 41 27
2 6 1463745 152177 0 0 0 0 0 0 2476 41847 28417 16 13 39 32
2 7 1463550 148351 0 0 0 0 0 0 2473 34091 26570 17 13 36 34
4 8 1463593 144694 0 0 0 0 0 0 2516 31392 24196 27 16 26 31
5 8 1464614 139329 0 0 0 0 0 0 2758 33268 29344 18 19 30 34
2 6 1464715 134423 0 0 0 0 0 0 2699 26064 29841 17 13 34 36
5 9 1464788 139100 0 0 0 0 0 0 2464 18065 24575 14 12 36 38
3 8 1464788 134384 0 0 0 0 0 0 2625 15827 30110 11 13 29 47
2 7 1464790 130033 0 0 0 0 0 0 2811 21990 28979 17 12 32 39
4 7 1464936 124708 0 0 0 0 0 0 2663 42988 31460 21 15 25 39
3 9 1465571 120795 0 0 0 935 9202 0 2818 46425 33260 20 18 22 40
4 9 1467351 116400 0 0 0 1243 46117 0 3322 45722 32458 18 17 21 44
2 7 1465766 117709 0 0 0 1111 71609 0 2334 57352 14443 23 13 26 38
3 7 1464908 117509 0 0 0 2729 6303 0 3101 38978 27375 43 21 18 18
3 7 1465483 116732 0 0 0 2262 5635 0 2517 37283 20649 30 14 28 28
2 7 1465876 116332 0 0 0 2280 22628 0 2417 28256 22816 20 13 30 38
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 7 1461513 120749 0 0 0 816 12165 0 2298 43533 14330 16 23 26 35
1 10 1463489 118674 0 0 0 1037 14676 0 2530 32953 17428 15 10 30 45
1 10 1462956 119265 0 0 0 1249 69037 0 2268 26853 16514 12 11 24 53
1 12 1462982 119224 0 0 0 2929 26019 0 2935 24471 34872 12 16 22 50
2 10 1463226 118995 0 0 0 3121 31686 0 3083 28326 41683 24 24 18 35
2 10 1463260 118968 0 0 0 2406 75458 0 2590 17752 26333 18 18 24 40
2 8 1463260 118963 0 0 0 645 2652 0 2100 16569 15367 16 7 38 39
1 8 1463260 118977 0 0 0 494 1748 0 1813 12472 12932 12 5 43 40
1 8 1463695 118582 0 0 0 921 2952 0 1832 15187 14932 9 8 42 41
5 10 1463510 118585 0 0 0 1231 5937 0 2319 44205 15756 14 17 27 42
1 9 1464034 117898 0 0 0 950 3347 0 2409 31598 15772 13 14 28 45
3 8 1464041 153910 0 0 0 148 436 0 2257 28270 17211 15 30 22 33
3 7 1463481 150684 0 0 0 0 0 0 2201 22733 18810 17 29 25 29
2 8 1463306 150219 0 0 0 0 0 0 1707 31664 7651 16 24 25 35
1 5 1463309 148564 0 0 0 0 0 0 1557 7161 10611 5 17 50 28
6 4 1463309 145430 0 0 0 0 0 0 1861 6222 16640 5 42 35 18
2 4 1462801 143079 0 0 0 0 0 0 1704 7839 17125 8 33 43 17
2 4 1462357 137966 0 0 0 0 0 0 2239 8996 28768 11 30 40 20
1 5 1462456 133674 0 0 0 0 0 0 2072 4598 24847 5 21 49 25
Greetings
Peter
We have a seriuos performance problem with our AIX-P-series server & AIX 4.3.3
The P-series is a S85 server with 8 CPU and 24 GB internal memory (and 6 GB Paging space)
We are using a Oracle 8.1.7 database and have around 1000 oracle connections.
The last month we are have more and more Paging to disk and have on the busy hours (10:30 + 15:30 CET) a 100 % CPU load
During to IBM this machine can handle arounf 2500 oracle connections....
So what is wrong with our settings?
Why do it already Page to disk?
For your info i will sent you some info from the machine:
#lsattr -E -l sys0 -a realmem
realmem 25165824 Amount of usable physical memory in Kbytes False
#lsps -s
Total Paging Space Percent Used
6144MB 24%
#lsps -a
Page Space Physical Volume Volume Group Size %Used Active Auto Type
paging01 hdisk1 pagevg 2048MB 24 yes yes lv
paging00 hdisk2 perfvg 2048MB 24 yes yes lv
hd6 hdisk0 rootvg 2048MB 24 yes yes lv
# vmstat 2
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 3 1470223 210508 0 0 0 1422 1282 0 414 1971 1500 88 94 99 59
3 7 1465485 214707 0 0 0 0 0 0 2848 31953 14591 30 18 26 27
3 8 1467528 209350 0 0 0 0 0 0 3217 40869 25871 27 22 23 28
3 8 1467812 204489 0 0 0 0 0 0 3082 28764 31617 24 22 26 28
2 7 1468104 198725 0 0 0 0 0 0 3011 18157 33753 19 14 30 36
4 7 1469511 192674 0 0 0 0 0 0 2993 30201 31152 19 26 28 27
3 7 1469538 190357 0 0 0 0 0 0 2162 20648 18488 13 21 32 34
2 6 1469998 189132 0 0 0 0 0 0 1979 21613 13117 14 19 39 29
3 5 1470587 187379 0 0 0 0 0 0 1947 23281 12188 15 18 36 31
2 6 1472608 184982 0 0 0 0 0 0 2734 51722 12550 21 22 38 18
5 7 1467856 188993 0 0 0 0 0 0 3088 78239 13191 20 35 23 21
3 6 1473987 176793 0 0 0 0 0 0 2783 25293 29238 20 36 22 23
3 5 1468310 176252 0 0 0 0 0 0 2961 28108 34162 19 34 23 24
6 7 1465294 174925 0 0 0 0 0 0 2946 38061 31417 18 34 24 24
4 5 1465111 173676 0 0 0 0 0 0 2368 28924 13782 29 30 18 23
5 6 1463706 173446 0 0 0 0 0 0 2135 25798 14247 29 28 23 21
6 5 1462929 175498 0 0 0 0 0 0 1944 20058 10553 28 21 29 22
6 5 1463532 188869 0 0 0 0 0 0 1731 23732 10750 24 24 32 21
7 6 1463086 185838 0 0 0 0 0 0 2570 19635 28689 17 32 31 21
3 6 1462752 182424 0 0 0 0 0 0 2605 17292 35752 15 31 28 27
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 10 1462565 177337 0 0 0 0 0 0 2867 11434 36568 11 19 33 36
3 5 1462554 172417 0 0 0 0 0 0 2590 17417 31772 13 14 38 36
6 8 1463786 166770 0 0 0 0 0 0 2516 32620 28999 17 22 36 25
1 4 1464677 161805 0 0 0 0 0 0 2659 24602 26877 17 12 45 25
2 7 1463744 156665 0 0 0 0 0 0 2523 31844 32554 16 15 41 27
2 6 1463745 152177 0 0 0 0 0 0 2476 41847 28417 16 13 39 32
2 7 1463550 148351 0 0 0 0 0 0 2473 34091 26570 17 13 36 34
4 8 1463593 144694 0 0 0 0 0 0 2516 31392 24196 27 16 26 31
5 8 1464614 139329 0 0 0 0 0 0 2758 33268 29344 18 19 30 34
2 6 1464715 134423 0 0 0 0 0 0 2699 26064 29841 17 13 34 36
5 9 1464788 139100 0 0 0 0 0 0 2464 18065 24575 14 12 36 38
3 8 1464788 134384 0 0 0 0 0 0 2625 15827 30110 11 13 29 47
2 7 1464790 130033 0 0 0 0 0 0 2811 21990 28979 17 12 32 39
4 7 1464936 124708 0 0 0 0 0 0 2663 42988 31460 21 15 25 39
3 9 1465571 120795 0 0 0 935 9202 0 2818 46425 33260 20 18 22 40
4 9 1467351 116400 0 0 0 1243 46117 0 3322 45722 32458 18 17 21 44
2 7 1465766 117709 0 0 0 1111 71609 0 2334 57352 14443 23 13 26 38
3 7 1464908 117509 0 0 0 2729 6303 0 3101 38978 27375 43 21 18 18
3 7 1465483 116732 0 0 0 2262 5635 0 2517 37283 20649 30 14 28 28
2 7 1465876 116332 0 0 0 2280 22628 0 2417 28256 22816 20 13 30 38
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 7 1461513 120749 0 0 0 816 12165 0 2298 43533 14330 16 23 26 35
1 10 1463489 118674 0 0 0 1037 14676 0 2530 32953 17428 15 10 30 45
1 10 1462956 119265 0 0 0 1249 69037 0 2268 26853 16514 12 11 24 53
1 12 1462982 119224 0 0 0 2929 26019 0 2935 24471 34872 12 16 22 50
2 10 1463226 118995 0 0 0 3121 31686 0 3083 28326 41683 24 24 18 35
2 10 1463260 118968 0 0 0 2406 75458 0 2590 17752 26333 18 18 24 40
2 8 1463260 118963 0 0 0 645 2652 0 2100 16569 15367 16 7 38 39
1 8 1463260 118977 0 0 0 494 1748 0 1813 12472 12932 12 5 43 40
1 8 1463695 118582 0 0 0 921 2952 0 1832 15187 14932 9 8 42 41
5 10 1463510 118585 0 0 0 1231 5937 0 2319 44205 15756 14 17 27 42
1 9 1464034 117898 0 0 0 950 3347 0 2409 31598 15772 13 14 28 45
3 8 1464041 153910 0 0 0 148 436 0 2257 28270 17211 15 30 22 33
3 7 1463481 150684 0 0 0 0 0 0 2201 22733 18810 17 29 25 29
2 8 1463306 150219 0 0 0 0 0 0 1707 31664 7651 16 24 25 35
1 5 1463309 148564 0 0 0 0 0 0 1557 7161 10611 5 17 50 28
6 4 1463309 145430 0 0 0 0 0 0 1861 6222 16640 5 42 35 18
2 4 1462801 143079 0 0 0 0 0 0 1704 7839 17125 8 33 43 17
2 4 1462357 137966 0 0 0 0 0 0 2239 8996 28768 11 30 40 20
1 5 1462456 133674 0 0 0 0 0 0 2072 4598 24847 5 21 49 25
Greetings
Peter