Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

How do I catenate over 400 files into one file? 1

Status
Not open for further replies.

tpbjr

MIS
Oct 8, 2004
120
US
I used the CAT command but after 300+ files the command blew up. Is there a way to iterate through a folder for given wild card (myfiles*.dat for example) and combine them into one file.

I posted this question in AWK originally but thought I better post it here for correctness:)

Thank you for all your help

Tom
 
You're probably running into the max command line length on your system.

Write a find that will locate all the files you want to find: 'find . -name \*.txt -print' and the do something like:
Code:
find . -name \*.txt -print | while read filename > destfile
do
   cat "$filename"
done
 
If you want to process files in a single directory (ie no sub directories) a simple ls will do:-

ls myfiles*.dat | cat >> newfile

HTH
 
Except that if he's hitting max length of the command line, then the shell will expand the *.dat glob and he'll have the same problem that he did using cat *.dat >> newfile.

The key with the find I posted is that \*.dat is protected from the shell and not expanded to the command line.
 
If the real problem is 'arg list too long':
(ls | grep 'myfiles.*\.dat' | xargs cat) > newbigfile

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ222-2244
 
RMBELGIUM:

First, we won't even talk about the Useless Use of Cat:

As ericbrunson suggests, what makes you think that `ls *.dat` won't end up with an 'arg list too long' error given enough files?

The strength of PHV's solution is that xarg guarantees not to overflow the buffer.

Happy New Year!
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top