File I/O and Nested AWK Search And Replace buffering issue

[mancocapac]

I have a script that builds a search string from the fields
in record. The Search string is made up of 3 parts depending
on the number of fields in the record. Some of the fields
in the record are files, which are themselves searched to
come up with a component of the overall search string. This
works fine and I am able to build my search string and ck that its good.

PROBLEM: For every record in my input file, I do a Search
and Replace on test.dat, using the Search String described above. My input file is about 3K records and
my test.dat is about 33K records. If I cut the inputfile down
to about 20 records everything works fine, but somewhere
between 20 and 3K it breaks, that is my test.dat doesn't
hold/get all the changes. I'm guessing its some type of
buffering/file-closing problem, but I don't know how to fix it.

Thanks for any and all help,
Tim

I am running on windows XP using the latest download of
cygwin

$ awk -f NestedAwkSandR.awk inputfile.csv

NestedAwkSandR.awk
BEGIN{
  FS="," 
  }
{
    [b]# I do a lot of processing of the record to come up with 
    # S3, S2, and S1 to get to this point, I left this
    # part out for simplification 
     [/b]

    # Run the Search and Replace
    srch = sprintf("'^C %s%s%s$' ", S3,S2,S1) # $ here is used for end-of-line 
    #printf("srch: %s\n", srch)

    repl = sprintf("'%s' ", $1)
    #printf("repl: %s\n", repl)
    # Nested AWK call
    # basically, i am trying to do 3,000 SandRs on the
    # test.dat file, one each per input record.
    [b]SrchReplCmd = ("awk -f SandR.awk -v srch="srch "-v repl="repl "test.dat > test.tmp")[/b]
    #print SrchReplCmd
    if ((SrchReplCmd | getline msg) > 0) {
      printf("SrchReplCmd msg:%s\n", msg) 
    }
    system("")  # do I need to flush before this move?
    close(SrchReplCmd)
    [b]move = ("mv test.tmp test.dat")[/b]
    system(move)
    close(move)
}

SandR.awk, make the change on line below search string
BEGIN{
#  printf("SandR: srch:=%s repl:=%s\n", srch, repl) 
}
  $0 ~ srch{
    print   # print the current $0
    getline # set $0 to next line
    gotone = 1
  }
  /^A VALUE=/{
    if(gotone){
#      printf("ORG: %s\n", $0)
      $0 = sprintf("A VALUE=%s", repl)
#      print 
    }
  }
  {print;gotone = 0;next}
#  {gotone = 0;next}

 

[PHV]

I'd replace this:
if ((SrchReplCmd | getline msg) > 0) {
with this:
while ((SrchReplCmd | getline msg) > 0) {

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[mancocapac]

PHV,
Thanks! That fixed it, but I don't understand why.

    while ((SrchReplCmd | getline msg) > 0) { 
#    if ((SrchReplCmd | getline msg) > 0) {
      printf("SrchReplCmd ERROR msg:%s\n", msg) 
    }

Could you please explain why this fixed the problem?
I capture the output and I never see the any ERROR msg.

Thanks