Help Using Sed/Awk for File Processing

[cdwells]

I have a data file such containing text such as below:

FIELD1~FIELD2~"FIELD3"~FIELD4-"$$$$"
FIELD1~FIELD2~"FIELD3"~FIELD4-"$$$$"
FIELD1~FIELD2~"FIELD3"~FIELD4-"$$$$"
FI
ELD1~FIELD2~"FIELD3"~FIELD4~"$$$$"

Some of the records are broken so I need to first remove all newline characters. Once this has been fixed, the characters ~"$$$$" will denote when my record line (will end) so I will need to replace all occurrences of ~"$$$$" with a newline character. My only problem is that I am not familiar with handling the newline character \n in sed or awk. Any suggestions?

 

[PHV]

Try something like this:

awk -F'~' '{buf=buf $0}
$NF=="\"$$$$\""{print buf;buf=""}
' /path/to/datafile

Hope This Help
PH.

 

[cdwells]

Thanks for the advice, however doesn't seem to be working. If anyone could give me advice as to stripping newline characters from a file using sed, that may steer me in the right direction (ie:

this
is
a
test
file

could become "this is a test file"

how would I go about accomplishing that (removing the newlines) using sed?

 

[PHV]

Tested with your 1st data example:

awk -F'~' '{buf=buf $0}
$NF~/"\$\$\$\$"$/{print buf;buf=""}
' /path/to/datafile

The problem was that some records have - instead of ~
Anyway, to strip ALL new-lines:

tr -d '\012' <inputfile

Hope This Help
PH.

 

[aigles]

Awk and sed possible solutions :

awk -F'~' '
buf=buf $0}
$NF~/-"\$\$\$\$"$/{ sub(/-"\$\$\$\$"$/,"",buf);print buf;buf=""}
' /path/to/datafile


sed -e '
:join
/-"\$\$\$\$"$/!{N
s/\\\n//
b join
}
s/-"\$\$\$\$"$//
' /path/to/datafile


Jean Pierre.

 

[PHV]

Forgot to get rid of trailing $:

awk -F'~' '{buf=buf $0}
$NF~/"\$\$\$\$"$/{
 print substr(buf,1,length(buf)-7);buf=""
}' /path/to/datafile

Hope This Help
PH.