Too many files in a directory

[moreCA]

Due to a script not doing its job I have run into the problem where I have over 40,000 files in a directory. Due to this I am unable to copy or mv anything over. Has any ever experienced this problem and if so how have you been able to resolve it w/o just getting rid of the directory the files are in.

Thank you...

 

[KenCunningham]

rm?

 

[PHV]

Which error have you with which mv command ?
I guess you have Arg list too long error, using some wildcard.
If this is a case:
man find
man xargs

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ222-2244

 

[SamBones]

Yeah, I've had this problem before. How to get around it is to not use any command where you have to put a wildcard character ([tt]*[/tt]) on the command line. This will just be expanded by the shell to too many parameters. You just have to get creative. Something like...

#!/bin/ksh

export COUNT=0

cd /problem_directory

ls -1 | while read FILE
do
    if [[ ${FILE} = @(*.log) ]]
    then
        # Put your pattern to match in the @() in the if above
        # This is assuming you only want to deal with some files
        rm ${FILE} ; # Or whatever you want to do to them
        print "Deleted: ${FILE}"
        (( COUNT += 1 ))
    fi
done

print "${COUNT} files deleted"

It's not a one liner, but it'll do the job.

Hope this helps.

 

[hfaix]

I'll usually write a for loop from a command line if I have too many files.

something like this ( don't run this unless you've tested it.)

for i in *.log
do
rm $i # test with ls $i
done

 

[piperent]

How about a simple:

find dir_name -name "*.log" -print -exec rm {}\;

It's only a one liner, and it should get the job done.

man find (for further explanation of parameters)

 

[stefanwagner]

commands operating on the current directory

du -s .

work, because the whole directory is used, but not the commandline expanded, as is if you use

du -s *

seeking a job as java-programmer in Berlin:

http://home.arcor.de/hirnstrom/bewerbung

 

[piperent]

You're absolutely right ..... I didn't consider the timing of variable substitution. Good call Stefan.

 

[piperent]

This script works correctly - just change the particulars to match the situation:


#!/bin/sh

LOGDIR=/u/sndrcv/xtest

find ${LOGDIR} -name "*.log" -print | while read LOGFILE
do
echo "File to process: ${LOGFILE}"
rm ${LOGFILE}
done

 

[hfaix]

The only thing I would be careful with, when using a "find", is that it can be recursive.

 

[moreCA]

I was able to resolve this issue.......

 

[KenCunningham]

.....how? Just for completeness, you understand.