Paging or not?

[unixfreak]

Our systems architect believes that paging is a bad thing and wants to prevent this on our servers by setting minperm and maxperm very low.

I think this is a bad idea but we raised a call with IBM who in their response said:
"Obviously paging memory out is not generally a good idea so it is advisable to try and prevent this"

I don't agree with this statement but what's peoples view on this? Is using paging space a bad idea or does it make memory management more efficient?

Cheers

 

[MoshiachNow]

I do not think paging is a matter of a "good" or "bad" idea - this is just how the AIX supposed to work.

If you have a huge amount of the physical memory - there will be practically no paging.
If there is not so much memory relatively to the running applications needs - there surely will be paging.

I think playing with minperm and maxperm is a bad idea that in reality will not improve performance big time,but can provide numerous support calls,if played wrongly.

Long live king Moshiach !

http://www.7for70.com/

 

[RodKnowlton]

If your memory is greater than the computational memory needs of your working set of applications, there are very few situations where the use of paging space is a performance gain.

"Playing" with minperm and maxperm (or any other VMM setting) is a bad idea, but making informed adjustments to them can improve performance big time for many loads. The default settings are those of a single user system.

Understanding what you're doing, and keeping track of settings so you can back out any change, should eliminate the need for support calls.

A good starting point for understanding the AIX VMM is Tom Farwell's articles on paging space for eServer Magazine. There are two, the link is to the first.

My response to the paging space question is: “I don’t want to use any paging space.” Why? Because, in general, if paging space is used, performance and throughput suffer. Therefore, it’s best to eliminate page-outs to paging space when possible.

His book is apparently out of print, but if you can locate a copy of RS/6000: Understanding Hardware, AIX Internals, and Performance: Professional Reference Edition, it'll give you even more of the nitty gritty detail.


Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L

 

[Chapter11]

Echoing everyone else, there is no One True Answer to how paging should be configured. It completely depends on the memory demands of the machine.

iirc, minperm and maxperm have to do with how much memory is set aside for file caching.

In my particular case, several of my machines run Oracle. On those machines, I have minperm and maxperm set low because Oracle should be handling caching of db data better than AIX can from a file level. On other machines where I have more file activity, those values are set higher.

 

[letis]

hello,

well, my point is that when the processor wants to access data from physical memory, it happens in nanoseconds, and when it has to seek it on disk, it takes milliseconds (x 1000), therefore it's a good idea to minimize disk access !!

regards,

 

[unixfreak]

But minimising disk access would mean paging out less frequently used pages wouldn't it? The problem we have is that with 8GB of memory and 2GB of paging space the servers sometimes dies completely because they run out of memory.
Our systems architect believes this is because it uses the memory for file caching and wants to stop this.

Cheers

 

[letis]

oups,

2GB of paging space is * not * enough for a 8GB memory system; AIX documentation states you should have at least around 8*1,25 of paging space (this is not the exact formula, I'm just rounding it here).

This total amount of paging can be splitted under different paging spaces (logical volumes), but they have to be equal in size, and located on different physical voulmes.

you can use topas (look for "non-comp memory" percent) to determine the percent of physical memory used for JFS filesystems caching. The client % is the percent of RAM used to cache NFS, CDRFS, or JFS2 filesystems.

By default AIX is not optimized to be an application server. For RDBMS purpose, minperm should be around 10/15% , and maxperm around 30%. If you use raw devices, these thresold have to be lowered again (5/10% for minperm, around 20% for maxperm).

These settings are not absolute, their exact value depend upon your workload.

regards,

 

[fluffybunny]

If your servers are dying, then it is most probably not your AIX OS that is the problem. It could be your application that is grabbing too much memory and leaking it out.
If I am not mistaken, AIX has a feature that prevents applications from grabbing too much memory, processor time, etc. You need to modify the requirements for your app to use more mem.

 

[RodKnowlton]

unixfreak,

Your paging space is definitely undersized. The default would be 8GB + 16MB, so either space has been removed or real memory has been added without adding paging space. If you have enough physical disks, I'd add three 2GB paging spaces, each on an independent disk. Keeping the spaces the same size makes the round-robin allocation work a little more efficiently.

When you say the server dies completely, do you mean the server application, or the entire machine/OS?

Tell your systems architect that you can't eliminate file caching in AIX. AIX actually uses the paging mechanism for reading files, and every AIX machine in the world (that's actually doing anything) will eventually have all memory that's not in use by applications filled with file pages. The good news is that all those pages are available for the VMM to steal until their percentage of real memory falls below minperm.

It sounds like either your applications are leaking memory or your real memory is undersized. If the latter, sizing paging space appropriately (which should be done no matter what) may fix the problem of crashes. If the former, it will likely only increase the average time to crash.




Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L

 

[screwloose]

You cannot determine that paging space is undersized. And you cannot state that it should be 8GB + 16MB. Plus, 2GB could possibly be plenty depending on the use of the machine.

Nowhere did I see vmstat, or computational and noncomputational data that would be used to determine the sizing of paging space. And this would not be a “one shot” gathering of data either. It should be ongoing and paging would be sized using historical data.

To check for memory leaks, use either svmon or ps.

 

[spamly]

Per a phone call about this very issue with IBM last friday:

To determine how much paging space should be allocated perform a "vmstat 5 1"
# vmstat 5 1
System Configuration: lcpu=12 mem=16384MB
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
2 7 1272430 1323223 0 0 0 137 527 0 1538 24138 606 2 2 88 8


Look at the number in the avm column. In this case "1272430".

Devide this number by 256.

In this case it is "4970 MB".

This number should be equal or less than total paging space.

 

[RodKnowlton]

screwloose,

You're absolutely right. I really shouldn't combine recovery from vacation, out of date documentation, and posting.

unixfreak,

Listen to screwloose regarding paging space sizing. Ignore my previous post's first paragraph. The rest is okay except for the parenthetical comment about changing your paging space.

spamly,

That's one strange procedure. For starters, the first line of vmstat output is summary, so "vmstat 5 1" is no different than "vmstat" without arguments. While most of the figures in that first line are averages since boot, the "avm" and "fre" figures are current values. So what the procedure says is that paging space should be greater than or equal to the size of your current working set, without taking into account the size of real memory, the variabilty of the working set, or how representative the current working set is. That's like deciding your monthly fuel budget based on how much you burn today, whether you rode mass transit to work or drove your Hum-Vee into the mountains. It'll be a number, but it could tie up a lot of resources unnecessarily or leave you stranded half way up a mountain some day.

Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L

 

[spamly]

Interesting analogy Rod.

I did kick off that command while the system was running "quietly", so you bring up a good point. Those were my notest during the phone conversation. Actually I ran it again during peak system utilization and I've added another almost 5GB of paging space.

One thing to note that if my system ever DID use all ~9 GB of paging space, my performance would be in the toilette. I typically see performance problems when start seeing paging in and I use more than about 1/2 GB page space.