Removing ^M from file 1

[confuseddddd]

Created a nawk script with a field separator of space and built an output file that has many records in the file and after each record of varying lengths, the record contains ^M. Want to be able to remove those carriage returns and the blank spaces that lead up to the ^M.
The script reads many header records, which are not sorted because all the detail follows each header record. Am splitting the input file into separate user files.

nawk 'BEGIN{
FS=" ";
PRE='`date -u +%Y%m%d`'
}
## get user id
if ( substr($1,1,4) == "HEADER" ) {
VAR=substr($2, 17, 3)
}
## build separate user files from input
print $0 >> PRE"_I01."VAR
}
END

I tried dtox and dos2unix but our UNIX system does not have those tools installed.

I tried sed '/^$/d' input > output and did not work
I tried grep -v "^[ ]*$" input > output and did not work
I tried awk 'length' input > output and did not work
I tried awk '/^[ ]*$/{next}{print}' input > output and did not work

Anyone have any other suggestions????

Thanks in advance.

 

[Ygor]

sed 's/^M//' input > output

Note: to get ^M press ctrl-V ctrl-M

 

[confuseddddd]

Okay, tried sed 's/^M//' input > output
and that worked great to get rid of ^M but I also need to remove the blank lines or areas that exist between each record...

Any ideas???

 

[Ygor]

Try...

sed 's/^M//' input | sed '/^$/d' > output

or change your awk program to only process non-blank lines...

nawk 'BEGIN{
FS=" ";
PRE='`date -u +%Y%m%d`'
}
[red]/./ {[/red]
## get user id
if ( substr($1,1,4) == "HEADER" ) {
VAR=substr($2, 17, 3)
}
## build separate user files from input
print $0 >> PRE"_I01."VAR
}
END

 

[CaKiwi]

Or do it all in the awk program. Get the ^M in vi by pressing cntl-V followed by cntl-M.

nawk 'BEGIN{
FS=" ";
PRE='`date -u +%Y%m%d`'
}
/./ {
[red]sub(/ *^M/,"&quot[/red]
## get user id
if ( substr($1,1,4) == "HEADER" ) {
VAR=substr($2, 17, 3)
}
## build separate user files from input
print $0 >> PRE"_I01."VAR
}' ...


CaKiwi

"I love mankind, it's people I can't stand" - Linus Van Pelt

 

[vgersh99]

faq80-1911

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[confuseddddd]

I tried all your suggestions and still am having a problem and its probably how I coded the processing....

I changed the nawk script to include the above suggestions:
}
/./ {
sub(\r,&quot;&quot
sub(\n,&quot;&quot
## get user id
if ( substr($1,1,4) == &quot;HEADER&quot; ) {
VAR=substr($2, 17, 3)
}
## build separate user files from input
print $0 >> PRE&quot;_I01.&quot;VAR
}' ...

The system does not like the sub statement..
awk: 0602-502 The statement cannot be correctly parsed. The source line is 88.
Any ideas on what I need to do to fix this parsing???

Also I did a od -hc on the file and saw at the end of each record is \r \n, so its not just ^M that is causing me the grief of extra spaces following the record.

 

[PHV]

Try thi:

sub("\r","")

Hope This Help
PH.