dos2unix convertsion on specific file types

[McQueen]

Hello all

I'm trying to get a script or command that will search down a specific directory looking for files that are of a file type of ascii or text and if they are run the dos2unix command on the file. I'm not sure where so start

Regards

McQUeen

 

[feherke]

Hi

find /the/dir -type f -exec sh -c "file -b '{}' | grep -q text && echo dos2unix '{}'" \;

If you like the output, remove the [tt]echo[/tt] and run it again.

Tested with [tt]bash[/tt] and [tt]ksh[/tt].

Feherke.

http://rootshell.be/~feherke/

 

[McQueen]

Hi feherke,

I ran the command and it came out with the following errors:-


find . -type f -exec sh -c "file -b '{}' | grep -q text && echo dos2unix '{}'" \;
grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
file: illegal option -- b
usage: file [-h] [-m mfile] [-f ffile] file ...
file [-h] [-m mfile] -f ffile
file -c [-m mfile]
grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
file: illegal option -- b
usage: file [-h] [-m mfile] [-f ffile] file ...
file [-h] [-m mfile] -f ffile
file -c [-m mfile]
grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
file: illegal option -- b
usage: file [-h] [-m mfile] [-f ffile] file ...
file [-h] [-m mfile] -f ffile
file -c [-m mfile]
grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
file: illegal option -- b
usage: file [-h] [-m mfile] [-f ffile] file ...
file [-h] [-m mfile] -f ffile
file -c [-m mfile]

etc

 

[feherke]

Hi

Yep, typical Unix... Try without those parameters :

find /the/dir -type f -exec sh -c "file '{}' | grep -w text > /dev/null && echo dos2unix '{}'" \;

Feherke.

http://rootshell.be/~feherke/

 

[KenCunningham]

Feherke, I'm intrigued, my (Solaris 8) man for grep gives the -w option as:

-w Search for the expression as a word as if surrounded by \< and \>.

What's the purpose of it here?

All I ask of you
Is make my wildest dreams come true

 

[columb]

You might be interested in this perl script which does exactly that. It uses the 'file' command to sort out which files need conversion. Certain file types are fixed by default (*.cbl, *.c, *.mk, *.sql) - see the 'not_data' subroutine.

My appologies about the lack of comments!

#!/bin/perl -w
use strict;

my $qflag = 0;

sub procdir ( $ )
  {
  my $dirname = shift;
  opendir DHANDLE, $dirname or return;
  foreach my $dentry ( readdir DHANDLE )
    {
    $dentry =~ /^\.{1,2}$/ and next;
    my $fullname = $dirname . '/' . $dentry;
    -f $fullname and 
        not_data ( $fullname ) and
        has_ctrl_m ( $fullname ) and 
        rm_ctrl_m ( $fullname );
    -d $fullname and &procdir ( $fullname );
    }
  closedir DHANDLE;
  }

sub not_data ($)
  {
  my $fname = shift;
  $fname =~ /\.cbl$/ and return 1;
  $fname =~ /\.mk$/ and return 1;
  $fname =~ /\.c$/ and return 1;
  $fname =~ /\.sql/ and return 1;
  my $ftype;
  `file -m /etc/magic $fname` =~ /:\s+(\w+)\s+/ and $ftype = $1 or  return
1;
  $ftype =~ /^gzip$/ and return 0;
  $ftype =~ /^PKZIP$/ and return 0;
  $ftype =~ /^data$/ and return 0;
  $ftype =~ /^compressed$/ and return 0;
  $ftype =~ /^executable$/ and return 0;
  $ftype =~ /^tar$/ and return 0;
  return 1;
  }


sub has_ctrl_m ( $ )
  {
  my $fname = shift;
  my $retval = 0;
  open FH, $fname or ( print "Unable to open $fname\n"), return; 
  while (<FH>)
    {
    /\r$/ and $retval = 1, last;
    }
  close FH;
  return $retval;
  }
  
sub rm_ctrl_m ( $ )
  {
  my $fname = shift;
  my $tmpname = $fname . ".tmp." . $$;
  $qflag or do
    {   
    print "$fname has ctrl-Ms. Do you want to fix \(y\/n\)\n";
    my $resp = <STDIN>;
    $resp =~ /^y/i  or return;
    };
  open IF, $fname or 
       ( print "Unable to open $fname\n"), return;
  open OF, ">", $tmpname  or 
       ( print "Unable to open $tmpname\n"), 
       close IF,
       return;
  while (<IF>)
    {
    chomp;
    s/\r$//;
    print OF $_, "\n";
    }
  close IF;
  close OF;
  rename $tmpname, $fname;
  print "$fname fixed\n";
  }

sub abort
  {
  print STDERR @_, "\n";
  print STDERR "Usage:- fixctrlMs.pl <directory> [y]\n";
  print STDERR "if the 'y' flag is used all corrections will be
automatic\n";
  exit -1;
  }

( $#ARGV != 0 ) && ( $#ARGV != 1 ) and abort "Invalid parameter count";
-d $ARGV[0] or abort "$ARGV[0] is not a directory";
$#ARGV == 1 && $ARGV[1] =~ /^y/i and $qflag = 1;
procdir ( $ARGV[0] );

Ceci n'est pas une signature
Columb Healy

 

[KenCunningham]

Hmm. I see the AIX man page is rather more succinct:

-w Does a word search.

All I ask of you
Is make my wildest dreams come true

 

[feherke]

Hi

You got it Ken.

First I used the [tt]file[/tt] command's -b option to exclude the file name from the output, so the [tt]grep[/tt]ed string was the file type description. Without that exclusion I want to ensure that a file with name img2text.tar.gz will not be considered text.

Feherke.

http://rootshell.be/~feherke/

 

[McQueen]

Hi all

Sorry I didn't get back to you sooner, I ran the find command on a directory with one file that I want the dos2unix convertsion program to run on as a test but it didn't remove the control charactrs from the file:-

$ cd /export/home/root/tmp/test_run
$ ls -al
total 10
drwxr-xr-x 2 root other 512 Sep 7 13:32 .
drwxr-xr-x 3 root other 1024 Sep 7 13:32 ..
-rw-r--r-- 1 root other 2414 Sep 7 13:32 jds.css
$ find . -type f -exec sh -c "file '{}' | grep -w ascii > /dev/null && echo dos2unix.sh '{}'" \;

But it didn't report any errors

Sorry to be a pain but I'm not sure what i'm going wrong

I ran the following command and it worked fine

$ find . -type f -name "*.css" -exec dos2unix.sh {} \;

McQueen

 

[PHV]

What is the output of the following command ?
file /export/home/root/tmp/test_run/jds.css

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[KenCunningham]

As Feherke says above, remove the echo if you're happy with what the command is going to do.

All I ask of you
Is make my wildest dreams come true

 

[McQueen]

I did remove the echo but still no luck,
and when I do a file one the file I get the following output

$ file jds.css
jds.css: ascii text

 

[PHV]

And what about this ?
find /export/home/root/tmp/test_run -type f |
while read f
do fgrep -q ascii "$f" && dos2unix.sh "$f"
done

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886