Removing Control Characters from a database dump 1

[Quinn71]

I have unloaded a database table to a pipe “|” delimited file. One of the columns in the table is a large char field. Within that column there are some records where an enter was used to format the field. The unload file shows the enter as a ^M and moves the rest of the column to a separate line in the unload file. My question is how can I remove the ^M from the top line and join the line beneath it to make a complete line? Most thankful for any advise.

 

[CaKiwi]

Try

awk '{gsub("\r",""print}' infile > outfile

or in vi enter

:%s/^M//g

where ^M is control-v followed by control-m CaKiwi

"I love mankind, it's people I can't stand" - Linus Van Pelt

 

[Quinn71]

Thanks CaKiwi. I have gone the vi route as well and it does a great job of getting rid of the ^M's. What it does not do is combine the next line with the line that had the ^M on it. Here is an example of my data.

Example 1 - This is correct.
column1|column2|column3 is a test

Example 2 - This is what is happening
column1|column2|column3 is^M
a test

The two lines in example 2 are treated as separate lines when I try to import the data.

Any additional help would be much appreciated.

 

[CaKiwi]

Maybe this will work if there is only 1 ^M in a record.

:g/^M/j

followed by

:%s/^M //
CaKiwi

"I love mankind, it's people I can't stand" - Linus Van Pelt

 

[iribach]

why not: dos2unix file file
see man pages

 

[iribach]

sorry cakiwi is correct

 

[Quinn71]

Unfortunately there are multiple ^M's in the file as well as multiple ^M's throughout the same record.

 

[CaKiwi]

I think if you keep repeating the 2 commands

:g/^M/j
:%s/^M //

until there are no more ^M characters in the file you will get what you want. CaKiwi

"I love mankind, it's people I can't stand" - Linus Van Pelt

 

[Quinn71]

You rock CaKiwi! You rock hard!!

Thanks for all your help.