how to append files without skipping header ?

[vikind]

Hi,

I have 2 types of files..the first type of file has a prefix adhoc_data_mmx.account.service.12152004.001
and the other type of file has a prefix data_mmx.account.serivce.12092004.001

now what i want here is each of the file has a header record in it. but if the file is a zero byte it will
not have a header record..so what i want to do is append the adhoc file to the corresponding data file
but without the header.

for example:

The data_mmx.account.service.12152004.001 file has 3 records it would look like

AGENT_CODE
100
200

The adhoc_data_mmx.account.service.12152004.001 has data for example say


AGENT_CODE
300
400

i have written a script that does some like this

sed '1d' adhoc_data_mmx.account.service.12092004.001 >> data_mmx.account.service.12152004.001

so the data_mmx.account.service.12152004.001 file will now contain the data

AGENT_CODE
100
200
300
400


(bascially it skips the header from the adhoc file and appends the data to the data file).

this works fine but when the actual data file for example has no data and the adhoc file contains data

then the data file would look like

300
400

basically the header information would be missing in this case..(how to avoid this)

how can i handle this thing ..in a script..can somebody help me out with this.

one more thing there are about 30 different files with adhoc_ as prefix and there are 30 corresponding
data files for it
e.g
adhoc..account will have a correspoding data..account.. file

similarly adhoc..insurance will have corresponding data..insurance.. file

so i want the script to work in a way such that once it finds with adhoc prefix it should find its corresponding

data file based on its pattern and then append the two files so that the final file for each of 30 file has header if either of the adhoc or data file contains data

 

[vgersh99]

man test

-s filename
True if filename exists and has a size
greater than zero.

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[vikind]

well..

its not just one file it has 30 files where each file name is unique..i was thinking if this can be doing using some loop rather than doing it individually for each of the file...

 

[vgersh99]

use '-s' to see if a file is of 0-length.

put a loop in place iterating through files with a common 'root' in their names.

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[futurelet]

Awk program [tt]conflate.awk[/tt]:

BEGIN {
  # Handle each filename.
  for (i=1; i<ARGC; i++)
  {
    file = ARGV[i]
    print file

    # We'll need the header from adhoc file unless
    # it's found in the main file.
    needheader = 1
    if ( (getline  <file ) > 0 )
      if ( length($0) > 0 )
        needheader = 0
    close(file)

    adhocfile = "adhoc_" file
    linenum = 0
    while ( (getline  <adhocfile ) > 0 )
    { linenum += 1
      if ( linenum>1 || needheader )
        print  >>file
    }
    close(file)
    close(adhocfile)

  }
  exit
}

Run with
[tt] awk -f conflate.awk data_mmx*
[/tt]
Let me know whether or not this helps.

If you have nawk, use it instead of awk because on some systems awk is very old and lacks many useful features. For an introduction to Awk, see faq271-5564.