tricky about grep command 3

[hokky]

Hi guys

I've got the file content like this called test

,,,,,,,,2007/05/14,1,0
,,,,,,,,2007/01/26,1,0
,,,,,,,,2007/01/26,2,0
,,,,,,,,2007/01/26,30,0
,,,,,,,,2007/01/26,61,0
,,,,,,,,2007/01/26,91,0
,,,,,,,,2007/01/26,122,0
,,,,,,,,2007/01/26,152,0
,,,,,,,,2007/01/26,183,0
,,,,,,,,2007/01/26,274,0
,,,,,,,,2007/01/26,365,0
,,,,,,,,2007/01/26,457,0
,,,,,,,,2007/01/26,548,0
,,,,,,,,2007/01/26,639,0
,,,,,,,,2007/01/26,730,0
,,,,,,,,2007/01/26,1096,0
,,,,,,,,2007/01/26,1461,0
,,,,,,,,2007/01/26,1826,0
,,,,,,,,2007/01/26,2557,0
,,,,,,,,2007/01/26,3652,0
,,,,,,,,2007/01/26,5479,0
,,,,,,,,2007/01/26,7305,0
,,,,,,,,2007/01/26,10957,0

if I type this

egrep -i ",0$" test

It didn't give me any results

but if I type this

egrep -i ",0.$" test

It does give me all the result above.

Can you guys explain to me why ?
Or is there something missing view in vi editor ?

Cheers guys,

 

[bigoldbulldog]

Does vi show something like ^M at the end of each line? Your terminal probably doesn't, but I'd bet your problem is a carriage return character at the end of each line. Has DOS or Windows touched this data? The '.' wildcard takes care of the issue in the egrep. Here's a possible fix

tr -d "\015" < test | egrep -i ",0$"

Cheers,
ND

 

[hokky]

Great bigoldbulldog !!!

Ok, how do I see the carriage return character ? Because I can't see it in vi or in my terminal

Have a star anyway.

 

[Annihilannic]

Try using cat -vet filename | more.

Annihilannic.

 

[kornShellScripter]

Hi hokky,

I'm developing a script that analyses non-unix files (that end up on a unix file system) so I've encountered exactly the same problem.

Commands like sed, awk and grep expect a file in unix format, so they expect their lines to end with a line feed.

I'm constantly looking into the values set inside my script's variables with "od -ta", for example:

$ VALUE=$(cut -c10-25 file) ; VALUE=${VALUE%%+( )} ; echo "$VALUE" | [blue]od -ta[/blue]
0000000   G   B  sp   1   2   3  sp   A   B   C  sp   G  lf

the way I've settled is using a trick like bigoldbulldog suggested.

Here's a couple of lines from my script:

OUT_NUM_PRINTED=$(grep "Records Printed :" $FILE_NAME | nawk '{print $8}' | tr -d '\015')

or, an alternative, which I used in a later version in my script:

OUT_NUM_PRINTED=$(tr -d '\015' <$FILE_NAME | nawk '/Records Printed :/{print $8}')

If I'm parsing the file and working on it line by line, I will remove replace all [cr] (the \015 or ^M value) values (in the [cr/lf] DOS end of line pair) with [lf] (which means my file would end up double spaced if read by a human, but to a script it doesn't matter):

cat $FILE_NAME | tr '\015' '\012' | while read j
   do
      <whatever>
   done

That last one could have been rewritten

tr '\015' '\012' <$FILE_NAME | while read j

but I'm trying to find a nice balance between being clever/efficient, and making my script read like english (reading left to right) as my script is reviewed by non-unix BAs and supported by non-unix production support staff.

 

[bigoldbulldog]

Just remembered, a lot of systems have a utility called dos2ux.

Cheers,
ND

 

[hokky]

Thanks guys,

One more question,
How can I find out about :

\015 means ^M
\012 means newline

Cheers,

 

[kornShellScripter]

Hi hokky,

man od

has a pretty neat table that I've been referring to since I started this role at the end of March just to keep a ready-reckoner of "Unix=lf, Dos=cr/lf, cr=\015=^M" - I didn't have to look those up as I wrote that so I guess it's stuck now

In my analysis of what the various text-hacking commands are producing (I'm using sed, grep, nawk and tr) I often pipe the output into "od -ta

 

[bigoldbulldog]

man ascii

has the table I use.

Cheers,
ND

 

[p5wizard]

$ echo "\r"|od -bc
0000000  [red]015[/red] [blue]012[/blue]
          [red]\r[/red]  [blue]\n[/blue]
0000002

HTH,

p5wizard