Hi, Guys, I have system perform 1

[llau]

Hi, Guys,

I have system performance problem when backuping DB to tapes (IBM 3583) using NBU4.5 and RMAN.

My system: 2 CPU/2 GB RAM with 4 GB swap;
OS: AIX5.1 (32bit) Application: Oracle8174(32bit)
The hot device is hdisk3, a RAID 0+1. I used only 1 channel to backup DB in RMAN, and the Netbackup buffer is 8 with 32K bytes.)

Firstly I got hign PageIn/PageOut in vmstat 10 results.
After the setting of MINPERM & MAXPERM was changed (from -p 20 -P80 to -p 5 -P 20), the page in &out is nearly zero now.
I also used the vmtune -R 64 to increase page ahead read. Other parameters of vmtune are default.
But the CPU wait is still very high, almost kept above 80% when reading DB files. Is there any clue to solve the problem?
#lsps -s
Total Paging Space Percent Used
4096MB 3%
#vmstat 10
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
1 3 404782 38964 0 0 0 1736 8660 0 1671 1142 3612 2 8 5 85
1 3 405393 38355 0 0 0 1729 23493 0 1738 2086 3757 3 9 4 84
1 3 404782 38969 0 0 0 1763 23612 0 1877 7998 4033 2 9 6 83
0 3 405396 38353 0 0 0 1739 5556 0 1712 1160 3696 2 8 5 84
1 3 404785 38963 0 0 0 1656 16720 0 1772 1802 3859 2 9 4 85
3 3 405400 38346 0 0 0 1755 3692 0 1726 1526 3725 2 10 6 83
2 3 404786 38958 0 0 0 1681 4702 0 1721 1114 3713 2 8 6 83
0 3 405399 38349 0 0 0 1694 21699 0 1768 2025 3807 2 9 4 85
1 3 404786 38964 0 0 0 1707 4707 0 1722 1117 3719 2 9 5 84
2 3 405861 37884 0 0 0 1525 41280 0 1487 1349 3213 3 8 8 81
0 1 403484 41230 0 0 0 18 60 0 330 3614 654 1 3 87 9
1 1 405341 38406 0 0 1 23 81 0 448 4305 808 3 4 80 13
1 2 404723 39026 0 0 0 23 82 0 353 3009 749 2 2 88 8
1 0 405336 38415 0 0 0 3 11 0 291 1464 517 1 1 95 2
0 1 405146 38601 0 0 0 15 45 0 291 2856 522 1 2 95 1
1 4 404751 38999 0 0 0 78 251 0 358 1694 669 1 2 92 4

Thanks in advance!
Laura

 

[Morsing]

What kind of tape device is it? And is it locally connected?

If it is a scsi tape this behaviour is normal and it caused by the fact that the tape is a slow sequential access device. The high wait percentage just means that the system wants to write to the device faster that it can process it.
Unless you can feel any slow down I wouldn't worry about it. The processer is free to do other work while the backup is waiting for the device to by ready again.

Cheers

Henrik Morsing
Certified AIX 4.3 Systems Administration
& p690 Technical Support

 

[comtec17]

llau,
Could you let me know how you changed the Maxperm, MinPerm, and vmtune? I am experiencing the same issues.

Thanks,

comtec17

 

[victorv]

hi,

if I have understood, your tape, is a LTO library and
is connected to the system in fiber-channel.

If the bottleneck is on tape, nothing you can do, but
probably you can tune network device buffer to cache
in memory data that will written on magnetic tape

bye

 

[llau]

1)To Comtec17

Issue this command:
#/usr/samples/kernel/vmtune -p 5 -P 20 -h 1
to reduce the file system buffer. By default, AIX uses -p 20 -P80.
(You may need some different values for your system.)

Also, maybe you need to put the following in /etc/inittab:
vmtune:2nce:/usr/samples/kernel/vmtune -p 5 -P 20 -h 1
for automatically change after system is restarted the next time.

2)We do use LTO(IBM 3583) and Fiber-channel, locally connected. I tried to use larger buffers for Veritas Netbackup (132K with 32 buffers, default 32K with 8 buffers). But I have to wait for the results.

Thanks,
LLAU

 

[bi]

comtec17: No one can make a blanket statement about what the vmtune parms should be on a box without knowing what kind of system it is, what kinds of disks and IO cards are being used, what applications are running on the system, how much memory, etc. So, just because llau's settings work for him, they might not work for you.

IBM has a Red Book on improving performance. You can go to

http://www.redbooks.ibm.com

and search for performance AND AIX.