fo colume in vmstat?

[rs6000er]

hi, all

I got a question about the "fo" colume in output of vmstat.
the concept of "fo" or "fi" si File page-ins/outs per second.
the out put of nmon show the memory like the following.
nmon v9a Hostname=ONHNAProd Refresh=1.0secs 13:11.17
Memory Use Physical Virtual Paging pages/sec In Out VM parameters
% Used 89.5% 0.1% to Paging Space 0.0 0.0 numperm 38.8%
% Free 10.5% 99.9% to File System 0.0 9.0 minperm 9.6%
MB Used 7327.9MB 12.4MB Page Scans 0.0 maxperm 19.1%
MB Free 864.0MB 10419.6MB Page Cycles 0.0 minfree 480
Total(MB) 8191.9MB 10432.0MB Page Reclaim 0.0 maxfree 1504

does it mean the "fi" or "fo" is the number of pages which is used by filesystem when the bufffer of filesystem is not enough for normal I/O.

in my production environment, the "pi" and "po" and "fi" are zero but "fo", and the default filesystem buffer tuning paramter (numfsbufs) is being considered to make more buffer for filesystem.

does anyone have the same experience?

any idea will be appreicated!

thanks in advance!!

Frank

 

[RodKnowlton]

does it mean the "fi" or "fo" is the number of pages which is used by filesystem when the bufffer of filesystem is not enough for normal I/O.

No, File pages in/out are the normal file I/O activity. AIX uses the same mechanism for reading/writing files as it does for virtual memory. If you're monitoring disk activity, then there's no real reason to care about "fi" or "fo", if they're what you say they are. What version of AIX are you running?
(I don't have "fi" and "fo" in 5.1).

I sometimes think that AIX provides those file paging stats so that admins will confuse them with virtual memory paging and think that it's running incredibly fast in spite of "thrashing".

Has anyone found a good use for them?

To see if numfsbufs needs to be increased, monitor the value of fsbufwaitcnt in "vmtune -a"'s output over time.

Rod Knowlton
IBM Certified Advanced Technical Expert pSeries and AIX 5L
CompTIA Linux+
CompTIA Security+

 

[rs6000er]

thank you for relay firstly!

my aix version is 5.1 with ML05.
the output of vmstat which has the "fi" and "fo" is with the flag I, this parameter also provides the extra colume p in kthr group which can be used to monitor the raw I/O (our production oracle database used the raw device).


Frank

 

[p5wizard]

fi and fo is paging activity due to file I/O (paging between memory and filesystems on disk

pi and po is paging activity due to processes being swapped between memory and PgSpace on disk


HTH,

p5wizard

 

[rs6000er]

P5wizard,

could you explain a bit about "fi and fo is paging activity due to file I/O (paging between memory and filesystems on disk)"? how do they work?

my understanding is that the busy filesystems are running out of the buffer in memory and have to swap some I/O pages which stay in memory normally into filesystem, when the read operation is heavy, the "fo" will show up, in reverse, the "fi" will be non-zero, is it correct?

thanks,

Frank

 

[rs6000er]

I copied something for vmstat as the below. it might be helpful for analyzing.

2 1 0 1001073 680399 0 11 0 0 0 0 1853 14302 2289 3 3 88 6 07:20
7 1 0 994330 687062 0 42 0 0 0 0 2804 23536 4049 7 5 84 5 07:25
1 1 0 996222 685044 0 23 0 0 0 0 3257 20646 4412 8 4 70 18 07:30
3 1 0 1000955 680058 0 26 0 0 0 0 3786 24073 5100 9 5 71 15 07:35
3 1 0 999550 681285 0 22 0 0 0 0 3791 22361 3890 7 4 76 12 07:40
2 1 0 1019908 628260 0 134 0 0 0 0 2970 20174 3774 7 4 83 6 07:45
3 1 0 1055148 592752 0 43 0 0 0 0 4153 26652 5143 14 6 67 14 07:50
4 1 0 1058778 588984 0 25 0 0 0 0 3732 25666 4897 10 5 77 8 07:55
3 1 1 1054379 593164 0 27 0 0 0 0 5467 34683 7815 15 7 58 20 08:00
5 1 1 1023364 624006 0 33 0 0 0 0 6045 40059 8955 13 8 60 19 08:05
8 1 1 1024124 622652 0 61 0 0 0 0 7598 48829 9314 23 10 46 21 08:10
6 1 2 1023192 623275 0 37 0 0 0 0 6546 39326 8032 15 9 41 35 08:15
4 1 2 1007276 606308 0 148 0 0 0 0 6468 41443 8011 18 9 39 34 08:20
7 1 2 1041695 571656 0 46 0 0 0 0 6403 40197 8377 17 9 41 33 08:25
6 1 2 1079066 533972 0 56 0 0 0 0 8730 55469 10301 33 11 25 30 08:30
5 1 2 1080202 532512 0 42 0 0 0 0 7612 46556 9531 22 10 30 37 08:35
5 1 2 1069718 510180 0 147 0 0 0 0 6384 39508 7288 20 8 38 34 08:40
10 1 2 1043384 536043 0 57 0 0 0 0 8374 55992 10617 21 12 34 32 08:45
5 1 2 1081981 497213 0 54 0 0 0 0 7599 47271 9421 23 10 35 32 08:50
8 1 2 1088765 489979 0 45 0 0 0 0 7308 49404 9098 25 10 34 31 08:55
4 1 3 1085383 460394 0 155 0 0 0 0 7064 48542 10003 24 10 30 36 09:00
7 1 2 1035743 509595 0 51 0 0 0 0 6807 47854 8905 20 10 33 37 09:05
4 1 1 1038434 506721 0 32 0 0 0 0 6041 37284 7863 15 8 44 34 09:10
5 1 1 1028598 516192 0 49 0 0 0 0 6562 42498 7833 23 9 39 30 09:15
9 1 2 1022276 489694 0 149 0 0 0 0 7499 45050 9383 21 10 37 32 09:20
7 1 2 1024207 487361 0 70 0 0 0 0 7749 53449 9519 25 12 33 30 09:25
6 1 2 1071942 439315 0 71 0 0 0 0 8098 50726 9441 27 11 26 35 09:30
4 1 2 1059679 451413 0 36 0 0 0 0 6822 42521 8758 18 9 33 40 09:35

 

[p5wizard]

fi : file pages that are read from disk into memory to satisfy one or more read's
fo : updated file pages in memory are written back to disk after one or more write's are sync'd to disk

your fi is always zero, implying that no reads from disk are taking place (everything is already in memory?), fo non zero means that updated file pages are being written to disk.

re numfsbufs:
numfsbufs is the number of buffers available per mounted filesystem. For every I/O to a JFS filesystem, LVM needs a free fsbuf structure. If there is none free, your I/O will be blocked.

see vmstat -v and look for filesystem I/Os blocked with no fsbuf

# vmstat -v|grep fsbuf
--> 3297 filesystem I/Os blocked with no fsbuf
0 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no fsbuf

If the number for this keeps rising, then it is a good idea to up numfsbufs with ioo - but you have to unmount and remount in order for the change to take effect.

there is a parameter for JFS, JFS2 (and NFS I think.)

numfsbufs is for JFS

HTH,

p5wizard

 

[rs6000er]

does AIX 5.1 have some relevant flag like -v which can be used to show the same contents?

the paramter -v olny is availiable in AIX 5.2 or higher.

the only thing I can remeber is the output of vmtune -a.

vmtune -a|grep wait
psbufwaitcnt = 1275543
fsbufwaitcnt = 18536
rfsbufwaitcnt = 0
xpagerbufwaitcnt = 0

is it same?

thank you, P5wizard!

Frank

 

[rs6000er]

this is my vmtune.
vmtune: current values:
-p -P -r -R -f -F -N -W
minperm maxperm minpgahead maxpgahead minfree maxfree pd_npages maxrandwrt
200552 401105 16 256 480 1504 65536 0

-M -w -k -c -b -B -u -l -d
maxpin npswarn npskill numclust numfsbufs hd_pbuf_cnt lvm_bufcnt lrubucket defps
1677709 83456 20864 1 245 1621 30 131072 1

-s -n -S -L -g -h
sync_release_ilock nokilluid v_pinshm lgpg_regions lgpg_size strict_maxperm
0 0 0 0 0 0

-t -j -J -z
maxclient j2_nPagesPer j2_maxRandomWrite j2_nRandomCluster
401105 32 0 0 1611497499

-Z -q -Q -y
j2_nBufferPer j2_minPageReadAhead j2_maxPageReadAhead memory_affinity
512 2 8 0

-V -i -e -E
num_spec_dataseg spec_dataseg_int jfs_clread_enabled jfs_use_read_lock
0 512 0 1

PTA balance threshold percentage = 50.0%

number of valid memory pages = 2097136 maxperm=20.0% of real memory
maximum pinable=80.0% of real memory minperm=10.0% of real memory
number of file memory pages = 721305 numperm=36.0% of real memory
number of compressed memory pages = 0 compressed=0.0% of real memory
number of client memory pages = 0 numclient=0.0% of real memory
# of remote pgs sched-pageout = 0 maxclient=20.0% of real memory

 

[rs6000er]

125701642 total address trans. faults
744873 page ins
8256362 page outs
0 paging space page ins
0 paging space page outs
0 total reclaims
57080447 zero filled pages faults
20169 executable filled pages faults
79229 pages examined by clock
0 revolutions of the clock hand
12373 pages freed by the clock
285474 backtracks
0 lock misses
0 free frame waits
0 extend XPT waits
110908 pending I/O waits
5369239 start I/Os
5369056 iodones
1042164131 cpu context switches
765576761 device interrupts
0 software interrupts
0 traps
5543451399 syscalls

 

[p5wizard]

Yes fsbufwaitcnt is the same

Imho your count is not really high. You upped it already from 197 to 245?

If you have a perf problem, then I would not look in to fi and fo, their values being what you posted earlier.

I guess the fo values are due to redolog archival, because your database is RAW, not JFS.

You still have memory free, perhaps you need a larger SGA and increase your oracle buffer quality (cache hit ratio).



HTH,

p5wizard

 

[rs6000er]

thank you so much for your excellent answers!

I will keep working on it.


thanks again!


Frank