topas says disks 0% busy, writing 2000 KBPS??? 1

[bi]

We are running tests on a large database that has been running very slowly. We are trying to speed it up. AIX is 4.3.3 and it is an Oracle database 8.1.7. We've got lots of memory (8 GB) and 6 CPUS @ 500 MHz. Paging is about 5%.

When the DBA started running his tests, I started topas to watch disk activity in particular. Things were going along normally. Then, all of a sudden, all of the disks (I had 12 of them displayed) went to 0% busy, but topas also said they were writing thousands of KBPS. This went on for about 5 minutes and even continued after the DBA stopped his test.

I suspect that the syncd may have kicked in, but I did not see that process in the process list in topas. I changed the vmtune maxrandwrt parameter from the default of 0 to 128 and we got better performance. What would a "typical" value be for this parameter? And has anyone ever seen disks being 0% busy but writing like crazy for about 3 to 5 minutes?

 

[avadh]

Hello,
Have you looked at error log. Have you done tuning for ORACLE. ORACLE has document on tuning ORACLE for AIX. Are you using Asynch IO, vmtune -> minperm, maxperm, minfree, maxfree and maxpageahead.

 

[bi]

Hi. Yes. we have finally turned on async and I have implemented some of the tips on vmtune you suggested (I think) in an earlier string I started. We are also going through the list of things in the redbook on tuning for Oracle.

Here are my vmtune settings:
minperm: 419223
maxperm: 1676892
minpgahead: 2
maxpgahead 448
minfree: 320
maxfree: 768
pd_npages: 524288
marandwrt: 256
maxpin: 1677713
npswarn: 65536
npskill: 16384
numclust: 2
numfsbufs: 93
hd_pbufcnt: 416
lvm_vufcnt: 9
lrubucket: 131072
defps: 1

and everything else is 0.

 

[gheist]

it takes time for aix kernel to calibrate disk usage so top* will show 0% right after you change a kernel parameter

 

[MoshiachNow]

Hi,

I used to use the following script to find the best params for the vmtune and schedtune.
The script runs any given "APPLICATION" in loop,while changing some vmtune params,displaying the results on the screen and as well,writing to log.
After every change,it loads the original vmtune params,to reset the situation.

in the "timex APPLICATION 0 2>$TIMEX_LOG" change the "APPLICATION" to your application in question path.
One can also add aaditional params that he wants to change.
===============================================
#!/bin/ksh
#The scripts runs the performance benchmark,while changing vmtune and other parameters,one at a time.It prints the vmtune params and the related benchmarks results into a log file.
#It activates the benchmark1.sh as a main benchmark
#Default vmtune values loaded on boot:
#/utils/admin/sci_admin: /usr/samples/kernel/vmtune -p 10 -P 30 -c 1 -R 64 -f 120 -F 184"

SCI_TEMP="/scitex/version/temp"
PERFTUNE_LOG="/$SCI_LOGS/perftune.log"
BENCHMARK_LOG="/scitex/version/logs/BENCHMARK.log"

HOSTNAME=`hostname`
MEMORY=`bootinfo -r`
MODELNAME=`lsattr -El sys0 | grep modelname |awk '{ print $2 }'`
PAGINGSPACE=`lsps -a -s | grep MB |awk '{ print $1 }'`
DATE=`date`
NOOFPROCS=`lscfg | grep proc | wc -l|awk '{ print $1 }'`
integer PERCENT=0
integer DEFAULT=5000
integer AVERAGE=0

RESET_VMTUNE ()
{
#reset to default vmtune params:
/usr/samples/kernel/vmtune -p 10 -P 30 -c 1 -R 64 -f 120 -F 184 -r 2 -b 93 >/dev/null
}

RESET_SCHEDTUNE ()
{
#reset to default vmtune params:
/usr/samples/kernel/schedtune -D >/dev/null
}

#Initialize variables:
RESET_VMTUNE
RESET_SCHEDTUNE
VMTUNE=`/usr/samples/kernel/vmtune`
SCHEDTUNE=`/usr/samples/kernel/schedtune`
TIMEX_LOG='/tmp/timex.log'
touch $BENCHMARK_LOG

RUN_BENCHMARK ()
{
#Runs each eteration 3 times and averages the result,displaying percentage relevant to Default values run
COUNT=3
AVERAGE=0
while (( $COUNT != 0 ))
do
timex APPLICATION 0 2>$TIMEX_LOG
#Get the run time
integer REAL=`grep real $TIMEX_LOG |awk '{ print $2 }'`
echo "|$REAL\c" >> $PERFTUNE_LOG
AVERAGE=$AVERAGE+$REAL
COUNT=$COUNT-1
fi
done
#Calculate the average of the 3 runs and it's percentage versus the default run
AVERAGE=$AVERAGE/3
#Set PERCENT to 100 for the default run
if [[ $PERCENT -ne 0 ]]
then
PERCENT=`echo "scale=4;$AVERAGE*100/$DEFAULT"|bc`
else
PERCENT=100
fi
#Establish the used paging space
usedPS=`lsps -a|grep rootvg|awk '{ print $5 }'`

echo "|| Average time = $AVERAGE seconds ,$PERCENT%", Used PS=$usedPS%>> $PERFTUNE_LOG
RESET_VMTUNE
RESET_SCHEDTUNE
}

#MAIN

#copy the last log file to xx.001
cp $PERFTUNE_LOG $PERFTUNE_LOG.001
echo "===========================" > $PERFTUNE_LOG
echo "$HOSTNAME , $MODELNAME , No of PROCS = $NOOFPROCS , Memory =$MEMORY , VER=$VERSION" , PAGINGSPACE=$PAGINGSPACE , $DATE >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
echo "VMTUNE DEFAULT PARAMETERS:" >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
echo "$VMTUNE" >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG

#Running with a default values:
RESET_VMTUNE
RESET_SCHEDTUNE
echo "-----------" >> $PERFTUNE_LOG
echo "##########################################" >> $PERFTUNE_LOG
echo " Running with a default values" >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
printf " %-3s %-3s " ' ' ' ' >> $PERFTUNE_LOG
RUN_BENCHMARK
#keep the Default time (run with default params) for percentage calculation
DEFAULT=$AVERAGE
echo "##########################################" >> $PERFTUNE_LOG

#Changing -P = maxperm values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -P=maxperm values in range 40/60/80%" >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for P in 40 60 80
do
printf " %-3s %-3s " "-P=" "$P" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -P $P >/dev/null
RUN_BENCHMARK
done

#Changing -r = minpgahead values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -r = minpgahead values in range 8/16/32 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for r in 8 16 32
do
printf " %-3s %-3s " "-r=" "$r" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -r $r >/dev/null
RUN_BENCHMARK
done

#Changing -R = maxpgahead values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -R= maxpgahead values in range 16/32/128 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for R in 16 32 128
do
printf " %-3s %-3s " "-R=" "$R" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -R $R >/dev/null
RUN_BENCHMARK
done

#Changing -f = minfree values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -f = minfree values in range 90/150/180 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for f in 90 150 180
do
printf " %-3s %-3s " "-f=" "$f" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -f $f >/dev/null
RUN_BENCHMARK
done

#Changing -F = maxfree values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -F = maxfree values in range 160/200/220 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for F in 160 200 220
do
printf " %-3s %-3s " "-F=" "$F" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -F $F >/dev/null
RUN_BENCHMARK
done

#Changing -c = numclust values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -c = numclust values in range 0 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for c in 0
do
printf " %-3s %-3s " "-c=" "$c" >> $PERFTUNE_LOG
/usr/samples/kernel/vmtune -c $c >/dev/null
RUN_BENCHMARK
done

####################################################################################
#Now we change the schedtune parameters:
echo " SCHEDTUNE DEFAULT PARAMETERS:" >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
echo "$SCHEDTUNE " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG

#Changing -f = TICKS values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -f = TICKS values in range 20/30/40 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for f in 20 30 40
do
printf " %-3s %-3s " "-f=" "$f" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -f $f >/dev/null
RUN_BENCHMARK
done

#Changing -d = SCHED_D values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -d = SCHED_D values in range 8/32 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for d in 8 32
do
printf " %-3s %-3s " "-d=" "$d" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -d $d >/dev/null
RUN_BENCHMARK
done

#Changing -r = SCHED_R values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -r = TICKS values in range 8/12/20 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for r in 8 12 20
do
printf " %-3s %-3s " "-r=" "$r" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -r $r >/dev/null
RUN_BENCHMARK
done

#Changing -m = MULTI values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -m = MULTI values in range 4/6/8/10 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for m in 4 6 8 10
do
printf " %-3s %-3s " "-m=" "$m" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -m $m >/dev/null
RUN_BENCHMARK
done

#Changing -w = WAIT values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -w = WAIT values in range 2/4 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for w in 2 4
do
printf " %-3s %-3s " "-w=" "$w" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -w $w >/dev/null
RUN_BENCHMARK
done

#Changing -e = GRACE values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -e = GRACE values in range 1/4 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for e in 1 4
do
printf " %-3s %-3s " "-e=" "$e" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -e $e >/dev/null
RUN_BENCHMARK
done

#Changing -t = MULTI values:
echo "##########################################" >> $PERFTUNE_LOG
echo " Changing -t = TIMESLICE values in range 2/4 " >> $PERFTUNE_LOG
echo "-----------" >> $PERFTUNE_LOG
for t in 2 4
do
printf " %-3s %-3s " "-t=" "$t" >> $PERFTUNE_LOG
/usr/samples/kernel/schedtune -t $t >/dev/null
RUN_BENCHMARK
done

mail -s &quot;$MODELNAME- PERFTUNE&quot; <MAILADDRESS> < $PERFTUNE_LOG



&quot;Long live king Moshiach !&quot;

 

[MoshiachNow]

Ops..
disregard the following line in the script:

#It activates the benchmark1.sh as a main benchmark
&quot;Long live king Moshiach !&quot;