EOL Problems on a text file

[tbohon]

I have a file of approximately 300 records which seems to be missing the EOF marker and, try as I may, I can't get it added to the file.

When I try to use vi (the only editor available on the AIX 5.x box) to edit the file, I receive the following:

"fixed.er" [Last line is not complete] 1 line, 3975187 characters

Have tried tr, using cat, using awk (from a post here - caused a core dump!?!?!) ... and am out of ideas.

Any thoughts and/or guidance appreciated.

Tnx.

Tom

"My mind is like a steel whatchamacallit ...

 

[PHV]

How do you know the file has approximately 300 records without EOL marker ? It is fixed length ?
How is it created and populated ?

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[dstxaix]

Subject of the post says EOL. Your message says EOF. What are you trying to add? EOL or EOF?

 

[tbohon]

Record count is an estimate based on several years' worth of these files - it could be slightly off but not far.

File contains HL7 data from a hospital system and is created by our Hospital Information System (HIS) before being transmitted over the network.

"My mind is like a steel whatchamacallit ...

 

[tbohon]

My apologies - subject line is incorrect, I'm trying to fix the EOF.

"My mind is like a steel whatchamacallit ...

 

[PHV]

I'm trying to fix the EOF

echo '' >> fixed.er

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[columb]

The problem seems to be that vi is seeing it as one long line of 3975187 characters which is going to be a bit of a pig to edit.

There are a number of options. If you know your record delimiter you can change it to newline using tr, or, if the records are fixed length you could use fold. Either of these will give one record per line which is the Unix way, and which vi expects.

Ceci n'est pas une signature
Columb Healy

 

[tbohon]

PH, tried the echo - still doesn't give me more than one long line.

Thinking it might be an EOL problem after all, I did a hex dump of the end of the first record in the file and find a x'0D at the end of each logical record - exactly the same as the file from a previous day:

43 41 44 3B 32 30 30 37 30 37 30 32 [COLOR=red]0D[/color] 5A 50 52

which translates to

CAD;20070702
ZPR

I'm stumped - I can probably go through the entire file in an editor and save each line - or maybe write a quick Perl script to break it up and write it back out - but both of those will take time, unfortunately.

Appreciate any further suggestions!

Tom

"My mind is like a steel whatchamacallit ...

 

[PHV]

tr '\r' '\n' < fixed.er > fixed.new

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[p5wizard]

What makes you think there's no EOF marker?

vi has counted lines and characters and stops at count of 1 line and 3975187 characters

vi complains about the one line not being terminated by a NL (newline) character. It has counted 3975187 characters so there IS an EOF marker, otherwise vi would keep on counting until something would break (a buffer overflow, whatever).

What you're facing is a problem of the source system not terminating the individual records with a NL character, which is kind of a prerequisite if you want to process the thus generated file afterwards with unix tools like grep or awk or ...

Perhaps there is a record separator character but it is something different than NL?

I realize that the contents of the file is not something that you want to disclose, but perhaps this (5-line oneliner) may help; it hides names and numbers, but might give us some insight to the structure of the file's records:

dd if=fixed.er bs=1024 count=2 |\
 tr '[a-z]' 'a*' |\
 tr '[A-Z]' 'A*' |\
 tr '[0-9]' '0*' |\
 od -xc

copy/paste the output back here and let's have a look...


HTH,

p5wizard

 

[tbohon]

columb:

See my post immediately above. The record delimiter is a hex 0D which is the way it is normally - why my Perl script and the vi editor only see one line is confusing.

Or maybe there were too many fireworks last night and my lack of sleep is contributing ...

Tnx.

Tom

"My mind is like a steel whatchamacallit ...

 

[PHV]

Tom, did you try my suggestion stamped 5 Jul 07 14:03 ?
Seems that HL7 is a Mac stuff ...

 

[p5wizard]

Try this then:

tr '\0x0D' '\n' <fixed.er >better.er

not sure on how to specify hex 0D in tr though...



HTH,

p5wizard

 

[mrregan]

phv has the correct solution.

 

[tbohon]

PHV: Yes, tried it and now have each HL7 segment on it's own line ... not exactly what the receiving system wants/needs/can handle. Each segment should be separated by a x'0D - which they are - but the end of the file, which should have a x'0D x'0A only has the x'0D . I've double checked today's file with yesterday's and they have exactly the same format except for the missing x'0A.

Tom

"My mind is like a steel whatchamacallit ...

 

[PHV]

So, you should apply my suggestion stamped 5 Jul 07 13:26.