Cloning Speed.

[pmidds]

We have Networker 6.2 on Solaris. We have a jukebox with two LTO I tape devices.

Our full backup of approximately 500 Gb this weekend took 24 hours to back up and a further 55 hours to clone this data.

Looking at previous posts i've seen that due to multiplexing savesets over multiple volumes cloning can take longer than the actual backup.

Does anyone have any tips for speeding up the cloning process ? or should I push for the purchase of a couple more drives

Thanks in advance.

 

[605]

I doubt your first statement - NW 6.2 is a Windows only version. Anyway, it should not matter
with respect to cloning.

What you actually can do is improving the speed by running a post-backup command script
that does the following:

1. Use mminfo to find all save set IDs that have been created by running this group
and send these to a file
2. Run nsrclone using this file as input.

Although i have not looked through the archives here i am sure that your will find an
example.

 

[fazil1]

you need more drives and you need to to implement Disk backup if you regulary clone tapes with that much DATA.
bombay4U@hotmail.com

 

[eskolnik]

fazil1... come on thats no answer

I have slow cloning too, but is better since I started using the following sample script:

#!/usr/bin/ksh
# ################################################################################################
#AUTHOR: Ed Skolnik
#MODIFIED: 2003/02/10 Created
# 2003/03/06 Changed Clone pool from Default Clone to
# GIS Default Clone to allow to recycle to other pools and
# to allow recycle from other pools
# 2003/04/01 EIS Changed to look back 5 days for save sets that have not been cloned
# 2003/04/15 EIS Add Check for NO savesets to clone
# 2003/04/21 EIS Deleted Detail s of what's going to be cloned
# 2003/04/23 EIS Corrected Starting message
# Also echo start to nsr log
# 2003/05/12 EIS Changed sort order for cloning from -omo Media and Offset
# to -otmo Time, Media and Offset
# 2003/05/13 EIS Backout out 05/12 change
# 2003/07/01 EIS Before starting the Clone process make the I/P
# Volumes readonly
# After set them back to notreadonly
# This will keep backup's from trying to use the same volumes
# as what is being used for clone processing.
# 2003/07/11 EIS Check to see if nsrclone is already running if so exit
# exit with Information message
# Also Allow clone when save groups are running if before
# 6:00 AM
# 2003/07/14 EIS Change to only look back 1 day instead of 5 do to
# weekend Network issues
# 2003/07/17 EIS Change wait time to 7.5 minutes from 15
# backup 7/14 change, look back 5 days again
# 2003/0718 EIS If i/p parm = last leave i/p tapes in readonly mode
# The eject process will be modified to only eject used tapes
# in read only mode.
# 2003/07/21 EIS Still having cloning issues si were only cloning from
# 07/21/03 forward
# also added !ssrecycle to selectection
# 2003/08/19 EIS This script will 1st clone all savesets that are complete
# on the volumes, then it will reissue the nsrclone
# to process savesets that spand mult. volumes
# 2003/-8/25 EIS Changed wait time for loops
# 2003/08/27 EIS Changed date format to match Network Daemon.log
# 2003/09/03 EIS add NOT suspect to mminfo for selecting save sets to
# clone (!suspect)
# 2003/09/15 EIS Correct mminfo for volumes was omo should be -omo
# 2003/10/09 EIS Corected ck_nsrclone function to wait if nsrclone is
# running
# 2003/10/28 EIS Syntax error in while loop for checking for
# this script already running....
# 2003/12/01 EIS In order to debug the Networker loop when a tape error
# occurs during cloneing adding Debug code.
# dbgcommand
# 2003/12/08 EIS Added Display of PID for this Run
# Processing:
#
# ################################################################################################
# ########################################################################################
nsr_logline () {
echo $(date +"%m/%d/%y %H:%M:%S") $(basename $0) Version $script_mod $* PID $$ >> $nsrlog
}

ck_for_running_savegrp () {
CT=1
while [ `ps -ef|grep -c 'root .*savegrp'` -gt 1 ] ; do
if [ $CT -gt 20 ] ; then
echo "$(basename $0) $(date) Never started cloning because backups were running all night! nsr_logline Never started cloning because backups were running all night
$(ps -ef|grep 'root .*savegrp' ) " > $tmpmessage
ps -ef | grep -v grep | grep nsr >> $tmpmessage
if [ $(date +%H) -lt 6 ]; then
display_message I
exit 1
else
mailto='chidcgrp,root_page'
display_message A
exit 69
fi
fi
sleep 450
CT=`expr $CT + 1`
done
}
ck_for_nsrclone () {
CT=1
while [ `ps -ef | grep -v grep | grep -c nsrclone` -gt 0 -o `ps -ef | grep -v grep | grep -c $0` -gt 2 ]; do
if [ $CT -gt 60 ] ; then
echo "$(basename $0) $(date) Is already running " > $tmpmessage
nsr_logline nsrclone already running
ps -ef | grep -v grep } grep nsrclone >> $tmpmessage
ps -ef | grep -v grep | grep $0 >> $tmpmessage
echo " " >> $tmpmessage
grep $(basename $0) $nsrlog | tail >> $tmpmessage
display_message I
exit 16
fi
sleep 300
CT=`expr $CT + 1`
echo ....................................
echo "ck_for_nsrclone Debug $(date)"
ps -ef | grep -v grep | grep nsrclone
ps -ef | grep -v grep | grep $0
done
echo "ck_for_nsrclone Debug "
ps -ef | grep -v grep | grep nsrclone
}


display_message () {
case $1 in
A|a) message_level="ALERT";;
I|i) message_level="INFORMATION";;
W|w) message_level="WARNING";;
*) message_level="";;
esac
if [ -r $tmpmessage -a -w $tmpmessage ]; then
:
else
echo "$(basename $0) Error in calling display_message function "
exit 69
fi
echo "$(date) $0 $(uname -n) $message_level $mailto " >> $tmpmessage
mailx -s"$message_level $(basename $0) $(uname -n) $message_level" $mailto < $tmpmessage
cat $tmpmessage
rm $tmpmessage
}
# #################################################################################
# Main Line
# #################################################################################
. /.profile
#
script_mod=$(ls -l $0 | awk '{print $6,$7,$8}')
fromdate="08/07/03"
passdir='/var/tmp'
dtstamp=$(date +%h%d_%H%M%S)
nsrlog=/nsr/logs/daemon.log
#
lastrun="$passdir/$(basename $0)_lastrun"
#
tmpssidonly="/tmp/$(basename $0)_ssid_only.$dtstamp"
tmpssidonly_span=/tmp/$(basename $0)_spanssid_only.$dtstamp
tmpvolumes="/tmp/$(basename $0)_volumes.$dtstamp"
tmpmessage="/tmp/$(basename $0)_message.$dtstamp"
nsrdpid=$(ps -ef | grep -v grep | grep \/nsrd | head | awk '{print $2}')
minusq=""
nsrjb_rc=0
rcx=0
rcx_span=0
mailto='root'
last=$1
# #########################################################################
ck_for_running_savegrp
ck_for_nsrclone
echo "debug $(date) No clone or savegrps should be running"
ps -ef | grep nsrc
ps -ef | grep save
#
minusq="!suspect,!ssrecycle,!incomplete,pool=GIS Default 86,pool='Default',pool='GIS Default',location=STKL40,copies=1,sscreate>5 days ago"
#
echo "$(date +%m/%d/%y) $(date +%H:%M:%S) $(basename $0) Starting " > $tmpmessage
nsr_logline "Starting "

echo " mminfo -t${fromdate} -omo -q $minusq -r'ssid,volume(6),client(15),level,sscreate(19),name" >> $tmpmessage
mminfo -t${fromdate} -omo -q"$minusq" -r'ssid,volume(6),client(15),level,sscreate(19),name' > $tmpssidonly
mminfo -t${fromdate} -omo -q"$minusq" -r'volume' > $tmpvolumes

echo "Number of Savesets to be Cloned $(cat $tmpssidonly | wc -l) " >> $tmpmessage
#cat $tmpssidonly >> $tmpmessage
#
if [ $(cat $tmpssidonly | wc -l ) -eq 0 ]; then
echo "Warning No savesets to Clone " >> $tmpmessage
rm tmpssidonly
display_message W
exit 14
else
display_message I
fi
#
#mminfo -t${fromdate} -omo -q"$minusq" -r'ssid' > $tmpssidonly
cp /dev/null $tmpssidonly
cp /dev/null $tmpssidonly_span
mminfo -t${fromdate} -omo -q"$minusq" -r'ssid,sumflags' | tail +2 | while read ssid sumflag; do
sumflag1=$(echo $sumflag | cut -c1)
case ${sumflag1} in
"c")
# The complete saveset is on this volume, so do it 1st
echo $ssid >> $tmpssidonly
;;
"h")
# Only the header is on this volume
echo $ssid >> $tmpssidonly_span
;;
*)
;;
esac
done
#
if [ -a $tmpvolumes ]; then
nsrjb -v -u
for I in $(cat $tmpvolumes); do
echo "nsrjb -Y -oreadonly $I "
nsrjb -Y -oreadonly $I
if [ $? -ne 0 ]; then
nsr_logline "Unable to mark $I as readonly "
fi
done
if [ -s $tmpssidonly ]; then
echo "/usr/sbin/nsrclone -v -b'GIS Default Clone' -S -f $tmpssidonly "
dbgcommand -p $nsrdpid "Debug=(3)"
nsr_logline "Starting Clone of savesets contained on a single volume"
/usr/sbin/nsrclone -v -b'GIS Default Clone' -S -f $tmpssidonly
rcx=$?
dbgcommand -p $nsrdpid "Debug=(0)"
else
rcx=0
fi
if [ -s $tmpssidonly_span ]; then
echo "/usr/sbin/nsrclone -v -b'GIS Default Clone' -S -f $tmpssidonly_span "
dbgcommand -p $nsrdpid "Debug=(3)"
nsr_logline "Starting Clone of savesets that span volumes "
/usr/sbin/nsrclone -v -b'GIS Default Clone' -S -f $tmpssidonly_span
rcx_span=$?
dbgcommand -p $nsrdpid "Debug=(0)"
else
rcx_span=0
fi
if [ $rcx = 0 -a $rcx_span = 0 ]; then
echo "`date +%m/%d/%Y` " > $lastrun
echo "$(basename $0) Good Run of nsrclone start time $dtstamp" \
>> $tmpmessage
echo "I/p parm ${last:-no} " >> $tmpmessage
echo "Number of Savesets Cloned $(wc -l $tmpssidonly) " >> $tmpmessage
nsr_logline "Ended "
rm $tmpssidonly
rm $tmpssidonly_span
display_message I
else
echo "$(basename $0) Clone alert failed return code $rcx $rcx_span from nsrclone command " > $tmpmessage
echo "start time $dtstamp" >> $tmpmessage
nsr_logline "Failed $rcx $rcx_span"
cat $tmpssidonly >> $tmpmessage
rm $tmpssidonly
rm $tmpssidonly_span
display_message A
fi
else
echo "no clone"
fi
nsrjb -v -u
if [ ${last:-no} = "last" ]; then
:
else
for I in $(cat $tmpvolumes); do
echo "nsrjb -Y -onotreadonly $I "
nsrjb -Y -onotreadonly $I
done
fi
rm $tmpvolumes
# Clean up temp files over 3 days old
find /tmp -name "$(basename $0)*" -type f -mtime +3 -exec rm {} \;
exit 0
# ######################################### End Of Script #####################################


Ed Skolnik