arg list too long: what is the limit for ksh shell? 4

[bi]

I'm having to clean up log files from an application that went berserk. I've tried using find with xargs and exec, but keep getting arg list too long.

so now, I'm using rm with wild cards to cut down on the number of files included in the arg. My question is, at what point can I stop pruning the list by using this manual rm method and set off a find to remove everything?

It's my understanding this is a shell limit. I'm using Korn shell. Does it also vary by platform?

 

[gheist]

does the
find /temporary/direstory -name "*bad*file*" -atine +2 -exec rm {} \;
answer your question?
(try echo instead of rm for first time)

 

[pcunix]

The "find" above will work, but piping it to "xargs rm" ratherthan using "exec rm {}" can be much faster for large sets. Tony Lawrence
SCO Unix/Linux Resources

http://pcunix.com

tony@pcunix.com

 

[bi]

The number of files I have to remove is v-e-r-y large. I used this command:

find . -name a.log* | xargs rm -f

I got the response arg list too large.

My question is, what is the upper limit for a list? Does it vary by shell? Does it vary by unix platform? Does it vary by command (for example, is it easier to move a large number of files than remove them?)

 

[MikeLacey]

I remember reading that the limit is 255....

your command is trying to expand the filename list on the command line, rather than letting find do the work, try this

find . -name 'a.log*' | xargs rm -f Mike
______________________________________________________________________
"Experience is the comb that Nature gives us after we are bald."

Is that a haiku?
I never could get the hang
of writing those things.

 

[gheist]

find . -name 'name*' -print | xargs rm -f
can help for older Solaris

 

[pcunix]

"find . -name 'name*' -print | xargs rm -f
can help for older Solaris"

will help for any system. What the '''s do is keep the shell from expanding the wildcard.
Tony Lawrence
SCO Unix/Linux Resources

http://pcunix.com

tony@pcunix.com

 

[bi]

pcunix,

Regarding: What the '''s do is keep the shell from expanding the wildcard.


So would that mean that I wouldn't get the arg list too long message even if I had thousands of files? (I told you the app went berserk!) I'm using the ' ' and it appears to be working. But I'm down to about 90K now!

Thanks for the tip.

 

[pcunix]

With the '*' in quotes, the shell does not expand it, so you won't get the argument.

If you are just removing everything, then of course

find . | xargs rm -f

will work also, but then so would

cd ..
rm -rf thatdirectory

Tony Lawrence
SCO Unix/Linux Resources

http://pcunix.com

tony@pcunix.com

 

[Kenton]

Great so far.

This has fixed my problem with rm but not mv

find /archive -type f -name '*' -print | xargs -i mv -f {} /oldfiles

gives me the error
myfile.sh[19]: /sbin/mv: arg list too long

What am I doing wrong?

Thanks
Kenton

 

[Annihilannic]

The -name '*' is superfluous, you can leave that out.

Try adding the -l 64 parameter to xargs to reduce the number of filenames (to 64, for example) it passes to mv each time. Annihilannic.

 

[iribach]

alternative:
#!/bin/sh
for file in `find .....`
do mv $file $dir
done

 

[dblair]

find . -name 'filename' -exec rm {} \;
to remove..
find . -name 'filename -exec mv {} tonewdir/. \;
to move..

 

[AHinMaine]

> The "find" above will work, but piping it to "xargs rm" ratherthan using "exec rm {}" can be much faster for large sets.

Just out of curiosity, can anyone explain why it's faster?

My thought was that keeping it contained in one command would be faster than piping it to yet another command. --
Andy

http://www.nachoz.com

 

[Annihilannic]

Without xargs, find has to fork() and exec() an separate /usr/bin/rm process to remove each individual file, which causes significant system overhead.

With xargs it only needs to do this once for every X number of files, where X is the number of files it can fit on one command line. Annihilannic.