Coverting Perl Code to Unix Script

[jayjaybigs]

Hello All,

I have a Perl Code as below. Because my input file is so large, hence Perl code modification will take for ever. However, I need to convert this Perl code to Unix Script. Please Help to write unix script that will perform the same job as the Perl code below. Thanx a bunch everyone for your time.

NOTE: The perl code below take one record from a file put it into array, modify certain element of the array and put the record back into a new file.


#!/usr/bin/perl
use strict ;
#use warnings ;
use Date::Manip ;

my (@input) ;
my ($input_record) ;
my ($output_file);
my ($input_file);
my ($array_record);
my (@array_record);
my (@array) ;
my ($name);
my ($lchar);
my ($date);
my (@array_file);
my (@names);
my ($new_record);

$input_file = "ytd_100.txt";
$output_file = "ytd_100.dat";

#check input file
(-e $input_file) || die ("input file does not exist \n") ;
open(INPUT, "$input_file") || die ("could not open infile\n");

open(OUTPUT, ">$output_file") || die ("could not open outfile \n") ;

@array_file = <INPUT> ;
foreach $array_record (@array_file){
@array = split (",", $array_record) ;

#Start Modifying Here

if (( $array[4] == 1) && ( $array[5] eq "A")) {
$array[4] = "BCGEU 2" ;
$array[5] = "REGULAR FT"
}

if (( $array[4] == 1) && ( $array[5] eq "C")) {
$array[4] = "BCGEU 2" ;
$array[5] = "REGULAR PT"
}

if (( $array[4] == 1 ) && ( $array[5] eq "F")) {
$array[4] = "BCGEU 2" ;
$array[5] = "SHORT TERM"
}

if (( $array[4] == 1 ) && ( $array[5] eq "E")) {
$array[4] = "BCGEU 2" ;
$array[5] = "SHORT TERM"
}
if (( $array[4] == 2) && ( $array[5] eq "A")) {
$array[4] = "BCGEU 1" ;
$array[5] = "REGULAR FT"
}

if (( $array[4] == 2) && ( $array[5] eq "C")) {
$array[4] = "BCGEU 1" ;
$array[5] = "REGULAR PT"
}

if (( $array[4] == 2) && ( $array[5] eq "F")) {
$array[4] = "BCGEU 1" ;
$array[5] = "SHORT TERM"
}

if (( $array[4] == 4) && ( $array[5] eq "A")) {
$array[4] = "EXCLUDED" ;
$array[5] = "REGULAR FT"
}

if (( $array[4] == 4) && ( $array[5] eq "C")) {
$array[4] = "EXCLUDED" ;
$array[5] = "REGULAR PT"
}

if (( $array[4] == 4) && ( $array[5] eq "F")) {
$array[4] = "EXCLUDED" ;
$array[5] = "SHORT TERM"
}

if (( $array[4] == 5) && ( $array[5] eq "A")) {
$array[4] = "EXCLUDED" ;
$array[5] = "SUPPORT FT"
}

if (( $array[4] == 5) && ( $array[5] eq "C")) {
$array[4] = "EXCLUDED" ;
$array[5] = "SUPPORT PT"
}

if (( $array[4] == 5) && ( $array[5] eq "F")) {
$array[4] = "EXCLUDED" ;
$array[5] = "SUPPORT ST"
}
$array[3] = " ";
chomp(@array) ;

#$array[12]=sprintf("%.2f", $array[12]);
$array[12]=$array[12]*0.010 ;

#$stop modifying here
$new_record = join ',', @array;
print OUTPUT "$new_record\n";

}
close (INPUT) ;
close (OUTPUT) ;

 

[TrojanWarBlade]

Why do you want to convert it from perl to shell script?


Trojan.

 

[duncdude]

first of all - can i ask why you want to do convert a Perl script to a UNIX script?


Kind Regards
Duncan

 

[jayjaybigs]

Because my input file is running into hundreds of thousands records, if not a million records. It is my believe that Unix script will be faster.

 

[duncdude]

I don't think it will be


Kind Regards
Duncan

 

[jayjaybigs]

Could you help to convert it though, then I will be able to test the faster script/code. Thanks for your attention DuncDude

 

[duncdude]

Hi Trojan

why not just do this - to remove the necessity to populate an array with the massive number of records you have:-

while (<INPUT>) { # @array_file = <INPUT> ;
# foreach $array_record (@array_file){
@array = split (",", $_);

Kind Regards
Duncan

 

[TrojanWarBlade]

Shell script will be much slower than perl.
But your perl code looks to be poorly designed.
As an example, examine your code and then look at this (and I don't suggest for one second that this is where we finish!)

if ($array[4] == 1) {
   if ($array[5] eq "A") {
      $array[4] = "BCGEU 2" ;
      $array[5] = "REGULAR FT"
   }

   if ($array[5] eq "C") {
      $array[4] = "BCGEU 2" ;
      $array[5] = "REGULAR PT"
   }

   if ($array[5] eq "F") {
      $array[4] = "BCGEU 2" ;
      $array[5] = "SHORT TERM"
   }

   if ($array[5] eq "E") {
      $array[4] = "BCGEU 2" ;
      $array[5] = "SHORT TERM"
   }
}

A simple example but hopefully makes the point.
Could you post some examples of the input file records and maybe we can seriously improve the performance for you.



Trojan.

 

[TrojanWarBlade]

Here is an idea for you, see what you think:

#!/usr/bin/perl -w

use strict;

my %hash = (
                '1A' => [ "BCGEU 2", "REGULAR FT" ],
                '1C' => [ "BCGEU 2", "REGULAR PT" ],
                '1F' => [ "BCGEU 2", "SHORT TERM" ],
                '1E' => [ "BCGEU 2", "SHORT TERM" ],
                '2A' => [ "BCGEU 1", "REGULAR FT" ],
                '2C' => [ "BCGEU 1", "REGULAR PT" ],
                '2F' => [ "BCGEU 1", "SHORT TERM" ],
                '3A' => [ "EXCLUDED", "REGULAR FT" ],
                '3C' => [ "EXCLUDED", "REGULAR PT" ],
                '3F' => [ "EXCLUDED", "SHORT TERM" ],
                '4A' => [ "EXCLUDED", "SUPPORT FT" ],
                '4C' => [ "EXCLUDED", "SUPPORT PT" ],
                '4F' => [ "EXCLUDED", "SUPPORT ST" ],
);

while(<>) {
  chomp;
  my @array  = split /,/, $_, -1;
  my $key    = $array[4] . $array[5];
  my $arrref = $hash{$key} || undef;

  if(defined $arrref) {
    $array[4] = $arrref->[0];
    $array[5] = $arrref->[1];
  }
  $array[3]  = " ";
  $array[12]*=0.010 ;
  print join(",", @array), "\n";
}

Trojan.

 

[duncdude]

couldn't a search & replace do this?

... after all, you seem to be searching for "4:F" and replacing with "EXCLUDED:SHORT TERM"

does this make sense?


Kind Regards
Duncan

 

[TrojanWarBlade]

Note: Without data, I haven't been able to test that idea.


Trojan.

 

[duncdude]

Sorry Trojan - i wasn't asking you ;-) The question was intended for jayjaybigs... i hadn't actually seen your post!


Kind Regards
Duncan

 

[TrojanWarBlade]

hehehe
That's ok.
It's basically where I was going anyway.


Trojan.

 

[duncdude]

Cool. He's disappeared though... when he comes back he's gonna sh*t himself on the number of replies


Kind Regards
Duncan

 

[jayjaybigs]

Thanks everyone for your input, I will posst input data shortly.

 

[TrojanWarBlade]

ok, thanks.



Trojan.

 

[TrojanWarBlade]

So then Duncs, you don't like my code?



Trojan.

 

[jayjaybigs]

Dunc,

Search & replace will not do as I was different matches in elements 4 and 5. Thanx anyway

 

[TrojanWarBlade]

I think Duncs is right here.
Your source will have fields 4 and 5 adjacent to each other and therefore you can treat them as one unique field (as I have done in my example code).


Trojan.

 

[duncdude]

Trojan

Your code looks good to me!

... but if he can get away with a search/replace ?


Kind Regards
Duncan