the find command hang in the script? 1

[Nikolai086]

there is on script, and it inclueds a command

find . ! -name "*.Z" -mtime +1 |xargs -I{} compress {} 2>/dev/null

As there are a lot of files will be found out, more than 20000, this script will be run every half an hour, but someday I received a warning, the find command use 20% CPU

#ps -ef |grep find
root 32202 44070 106 16:35:24 - 1270:26 find . ! -name *.Z -mtime +1

>_<|||
Then I have to kill it, I think it's the find command's wrong.

Does any advice? how can I modify it so that resolve it forever.

Thank you!

--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.

 

[Annihilannic]

I don't think there's anything wrong with the find command, there must have been something else wrong on the system at the time. Does this happen every time you run it?

Annihilannic.

 

[Nikolai086]

I put this script in the cronjob.
and it will eat the CPU after a couple of days..

--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.

 

[Annihilannic]

Ah. What directory is '.'? Is there a cd command before the find command in the script?

20000 isn't a very large number of files. Maybe the filesystem is corrupt, have you tried unmounting it and running an fsck?

Try testing it outside cron and see how long the command takes to run.

Annihilannic.

 

[Nikolai086]

Ah. What directory is '.'? Is there a cd command before the find command in the script?

Yes

I set some log in the script, find something strange

#################################################
echo "script start at `date`" >>/abc/script.log
..................
..................
..................
echo "find start at `date`" >>/abc/script.log

find . ! -name "*.Z" -mtime +1 |xargs -I{} compress {} 2>/dev/null

echo "find end at `date`" >>/abc/script.log
..................
..................
..................
echo "script end at `date`" >>/abc/script.log

########################################################

Here is the log:

Starting at Thu Dec 27 16:30:00 BEIST 2007
End at Thu Dec 27 16:36:02 BEIST 2007
Starting at Thu Dec 27 16:36:02 BEIST 2007
End at Thu Dec 27 16:36:32 BEIST 2007
Script Start at Fri Dec 28 10:30:00 BEIST 2007
Script END at Fri Dec 28 10:30:03 BEIST 2007
Script Start at Fri Dec 28 11:30:00 BEIST 2007
Script END at Fri Dec 28 11:30:02 BEIST 2007
Script Start at Fri Dec 28 12:30:00 BEIST 2007
Script END at Fri Dec 28 12:30:02 BEIST 2007
Script Start at Fri Dec 28 13:30:00 BEIST 2007
Script END at Fri Dec 28 13:30:02 BEIST 2007
Script Start at Fri Dec 28 14:30:02 BEIST 2007
Script END at Fri Dec 28 14:30:07 BEIST 2007
Script Start at Fri Dec 28 15:30:01 BEIST 2007
Script END at Fri Dec 28 15:30:04 BEIST 2007
Script Start at Fri Dec 28 16:30:00 BEIST 2007
find Starting at Fri Dec 28 16:30:01 BEIST 2007
find End at Fri Dec 28 16:37:07 BEIST 2007
find Starting at Fri Dec 28 16:37:07 BEIST 2007
find End at Fri Dec 28 16:37:31 BEIST 2007
Script END at Fri Dec 28 16:37:33 BEIST 2007
Script Start at Fri Dec 28 17:30:02 BEIST 2007
Script END at Fri Dec 28 17:30:04 BEIST 2007

Why the there is no find log in script's start & end log

--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.

 

[Annihilannic]

I guess there must be some other logic in the script that controls whether it gets to the part where it runs find or not. Without seeing the entire script it's impossible for me to say.

Annihilannic.

 

[geirendre]

You could add more "debug-output" to your script.
like:

echo $PWD >> /abc/script.log

(show value of some variables while executing),
or:

echo "subroutine 1 ran OK." >> /abc/script.log

(mark that shows if part of script did indeed run)
in strategic places in the script to show how the
script runs.

 

[Nikolai086]

#Here is the script

DataPath=/backup/performance/data
ReportDIR=/data/workshop/report
ScriptDIR=/data/workshop/scripts
TIME=`date +%H`

cd ${ScriptDIR}

if [ $USER != 'root' ]
then
clear
echo '\n\n\n\n'
echo ' Please login root run this script'
echo '\n\n\n\n'
exit
fi

if [ $TIME -eq 16 ]
then

# Check filesystem utilization

df -k > ${DataPath}/df-`hostname`.`date '+%m%d%H%M'`

#
# Check process status
#

echo "There are total `ps -ef| wc -l | awk '{print $1}'` processes in system now." >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt

# Check top 10 process
echo "Top 10 process at `date` " >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt
ps ug | egrep -v "CPU|kproc" | sort +2b -3 -n -r \
| head -n 10 >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt

# Check online userID
echo " " >> ${ScriptDIR}/report.txt
echo "Online userID at `date`" >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt
who -u >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt
echo " " >> ${ScriptDIR}/report.txt

#
# Compress files
#
for i in `cat ${ScriptDIR}/compress.lst`
do
cd $i
find . ! -name "*.Z" -mtime +1 |xargs -I{} compress {} 2>/dev/null
done

cd ${ScriptDIR}
mail -s "`hostname` system status report `date` " abc@abc.com < ${ScriptDIR}/report.txt
# Rename ${ScriptDIR}/report.txt
mv ${ScriptDIR}/report.txt ${ReportDIR}/report.txt.`date +%m%d`
chown batch:qad ${ReportDIR}/report.txt.`date +%m%d`
fi

#
# Check flag file.
#

for i in /u/batch/mfgp/reserve
do
if [ -f $RESDIR/*session* ]
then
echo "Attention! there is a session still running in $i" >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
fi
done

#Check system defunct process
ps -ef | grep defunct | grep -v grep | sort -u > ${ScriptDIR}/defunct-new.txt
i=`diff ${ScriptDIR}/defunct-new.txt ${ScriptDIR}/defunct-old.txt | grep "<" |wc -l|awk '{print $1}'`
if (( $i > 0 ))
then
echo " " >> ${ScriptDIR}/warning.txt
echo "Warning!!!, system have new defunct process, please check it carefully!!!" >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
ps -ef|grep defunct | grep -v grep >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
mv ${ScriptDIR}/defunct-new.txt ${ScriptDIR}/defunct-old.txt
fi

for i in `cat ${ScriptDIR}/process.lst`
do
ps -ef|grep $i | grep -v grep > /dev/null
if [ $? -eq 1 ]
then
echo " " >> ${ScriptDIR}/sendmail.flag
echo "Attention!!!, $i does not run." >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
fi
done


# Check file system space
flag=`df -k | awk '{if($4>97) print $0}'|wc -l`
if [ $flag -gt 0 ]
then
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo "Space utilization over 90% :" >> ${ScriptDIR}/warning.txt
df -k | head -1 >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
df -k | awk '{if($4>90) print $0}' >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
fi

# system utilization summery
grep 0 $DataPath/vmstat.????.`date +%m%d%y` | tail -60 > ${ScriptDIR}/vmstattemp
line=`awk 'BEGIN {A=0;B=0;C=0;D=0;E=0;F=0;G=0;H=0;I=0;J=0};
{A=$6;B=$7;C=$14+$15;D=$2;E=$17};
A >=1 {F+=1};
B >=1 {G+=1};
C >=70 {H+=1};
E >=30 {I+=1};
D >J {J=D};
END {print NR,F,G,H,I,J}' ${ScriptDIR}/vmstattemp`
flag1=`echo $line | awk '{if($2>20) print $0}' | wc -l`
flag2=`echo $line | awk '{if($3>20) print $0}' | wc -l`
flag3=`echo $line | awk '{if($4>40) print $0}' | wc -l`
flag4=`echo $line | awk '{if($5>40) print $0}' | wc -l`
flag5=`echo $line | awk '{if($6>20) print $0}' | wc -l`
if [[ ${flag1} -gt 0 || ${flag2} -gt 0 || ${flag3} -gt 0 || ${flag4} -gt 0 || ${flag5} -gt 0 ]]
then
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
echo "Warnning! System is busy now, please check" >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
head -1 ${ScriptDIR}/title-sys.txt | awk '{printf "%10s %10s %10s %10s %10s %10s\n",$1,$2,$3,$4,$5,$6}' >> ${ScriptDIR}/warning.txt
echo $line | awk '{printf "%10s %10s %10s %10s %10s %10s\n",$1,$2,$3,$4,$5,$6}' >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
cat readme.txt >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
fi
rm ${ScriptDIR}/vmstattemp

errpt > ${ScriptDIR}/errpt.new
if [ `diff ${ScriptDIR}/errpt.new ${ScriptDIR}/errpt.old | wc -l | awk '{print $1}'` -gt 0 ]
then
echo "System error report:" >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
errpt >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
fi
errpt > ${ScriptDIR}/errpt.old

# Check system faild login
who -s /etc/security/failedlogin > failedlogin.new
if [ `diff ${ScriptDIR}/failedlogin.new ${ScriptDIR}/failedlogin.old | wc -l | awk '{print $1}'` -gt 0 ]
then
echo "As below is faild login record in `date +%h%y`" >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
diff ${ScriptDIR}/failedlogin.new ${ScriptDIR}/failedlogin.old | grep "<" | cut -d" " -f2- >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/warning.txt
echo " " >> ${ScriptDIR}/sendmail.flag
fi
mv ${ScriptDIR}/failedlogin.new ${ScriptDIR}/failedlogin.old
chown batch:qad ${ScriptDIR}/failedlogin.old

if [ -f ${ScriptDIR}/sendmail.flag ]
then
mail -s "`hostname` system warning report `date` " abc@abc.com < warning.txt
rm ${ScriptDIR}/sendmail.flag
cat ${ScriptDIR}/warning.txt >> ${ReportDIR}/warning.txt.`date +%m%d`
chown batch:qad ${ReportDIR}/warning.txt.`date +%m%d`
rm ${ScriptDIR}/warning.txt
fi

--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.

 

[Nikolai086]

The find command seems run once a day...
The log:

find Starting at Fri Dec 28 16:30:01 BEIST 2007
find End at Fri Dec 28 16:37:07 BEIST 2007
find Starting at Fri Dec 28 16:37:07 BEIST 2007
find End at Fri Dec 28 16:37:31 BEIST 2007
find Starting at Sat Dec 29 16:30:00 BEIST 2007
find End at Sat Dec 29 16:36:27 BEIST 2007
find Starting at Sat Dec 29 16:36:27 BEIST 2007
find End at Sat Dec 29 16:36:51 BEIST 2007
find Starting at Sun Dec 30 16:30:01 BEIST 2007
find End at Sun Dec 30 16:38:26 BEIST 2007
find Starting at Sun Dec 30 16:38:26 BEIST 2007
find End at Sun Dec 30 16:38:53 BEIST 2007
find Starting at Mon Dec 31 16:30:01 BEIST 2007
find End at Mon Dec 31 16:35:28 BEIST 2007
find Starting at Mon Dec 31 16:35:28 BEIST 2007
find End at Mon Dec 31 16:35:50 BEIST 2007
find Starting at Tue Jan 1 16:30:00 BEIST 2008
find End at Tue Jan 1 16:34:41 BEIST 2008
find Starting at Tue Jan 1 16:34:41 BEIST 2008
find End at Tue Jan 1 16:34:59 BEIST 2008

But the script run many times a dat:


Script Start at Tue Jan 1 16:30:00 BEIST 2008
Script END at Tue Jan 1 16:35:01 BEIST 2008
Script Start at Tue Jan 1 17:30:00 BEIST 2008
Script END at Tue Jan 1 17:30:02 BEIST 2008
Script Start at Tue Jan 1 18:30:00 BEIST 2008
Script END at Tue Jan 1 18:30:01 BEIST 2008
Script Start at Tue Jan 1 19:30:00 BEIST 2008
Script END at Tue Jan 1 19:30:02 BEIST 2008
Script Start at Tue Jan 1 20:30:00 BEIST 2008
Script END at Tue Jan 1 20:30:03 BEIST 2008
Script Start at Tue Jan 1 21:30:00 BEIST 2008
Script END at Tue Jan 1 21:30:02 BEIST 2008
Script Start at Tue Jan 1 22:30:00 BEIST 2008
Script END at Tue Jan 1 22:30:02 BEIST 2008
Script Start at Tue Jan 1 23:30:00 BEIST 2008
Script END at Tue Jan 1 23:30:02 BEIST 2008
Script Start at Wed Jan 2 00:30:00 BEIST 2008
Script END at Wed Jan 2 00:30:01 BEIST 2008
Script Start at Wed Jan 2 01:30:25 BEIST 2008
Script END at Wed Jan 2 01:31:15 BEIST 2008
Script Start at Wed Jan 2 02:30:00 BEIST 2008
Script END at Wed Jan 2 02:30:02 BEIST 2008
Script Start at Wed Jan 2 03:30:00 BEIST 2008
Script END at Wed Jan 2 03:30:02 BEIST 2008
Script Start at Wed Jan 2 04:30:01 BEIST 2008
Script END at Wed Jan 2 04:30:03 BEIST 2008
Script Start at Wed Jan 2 05:30:00 BEIST 2008
Script END at Wed Jan 2 05:30:02 BEIST 2008
Script Start at Wed Jan 2 06:30:00 BEIST 2008
Script END at Wed Jan 2 06:30:03 BEIST 2008
Script Start at Wed Jan 2 07:30:01 BEIST 2008
Script END at Wed Jan 2 07:30:02 BEIST 2008
Script Start at Wed Jan 2 08:30:00 BEIST 2008
Script END at Wed Jan 2 08:30:02 BEIST 2008

--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.

 

[Annihilannic]

Yes, that's correct. See the section that looks like this:

if [ $TIME -eq 16 ]
then
...
fi

That part only runs when the hour component of the time is 16, i.e. between 16:00 and 16:59.

Annihilannic.

 

[Annihilannic]

You should look at the contents of /data/workshop/scripts/compress.lst to figure out which filesystems the find command is running on. Is the problem still happening?

To debug it, try adding something to this section:

#
# Compress files
#
  for i in `cat ${ScriptDIR}/compress.lst`
  do
     echo "Compressing files in $i at `date`" >>/abc/script.log
     cd $i
        find . ! -name "*.Z" -mtime +1 |xargs -I{} compress {} 2>/dev/null
  done

Annihilannic.

 

[SamBones]

You may be trying to "compress" subdirectories if there are any. Adding "[tt]-type f[/tt]" to the find command will limit it to only compressing normal files.

Also, try collecting the output from the [tt]xargs[/tt] and [tt]compress[/tt] instead of throwing them into [tt]/dev/null[/tt]. There might be a helpful error message you are missing.

Try this...

#
# Compress files
#
  for i in `cat ${ScriptDIR}/compress.lst`
  do
     echo "Compressing files in $i at `date`" >>/abc/script.log
     cd $i
     find . [b]-type f[/b] ! -name "*.Z" -mtime +1 | xargs -I{} compress {} [b]>> /abc/compress.log 2>&1[/b]
  done

 

[Nikolai086]

Thanks all,
I will update any new error



--
Best Regards!
Hello, I am Nikolai NG, come from Carton, China. I love here.