Nested Variables

[BStopp]

I have a file which contains two columns. Column 1 is the pattern to match, column 2 contains the replacement pattern.

So:

B_(Stopp)_(\d{4})(\d{2})\d{2}   B$1_Y$2_M$3

i want to do substitution using the interpolated varabiles in my code. I can get the pattern to match just fine, but when i place the second part in the replacment string, Perl interpolates the variable the string is contained in, but it does not interpolate the $1, $2, $3 variables within that string.

$line =~ s/$changes{$oldname}/$changes{$newname}/g;
print $line;
exit;

The print statement always prints "B$1_Y$2_M$3", not the values that should be stored in the $1, $2, and $3 variables.

Anyone know what I'm doing wrong? Thanks!

 

[mikedaruke]

I couldn't get it to work either. But what I did which works but adds another step. I search for the search part first, then once I found it it populate the variables $1 $2, then I went for the search and replace.

#Variables
$line = 'abckdefgh123456';
$search = '(...).*?h(1.*?6)';

# Do the find first so the variables get populated
$line =~ m/$search/gs;
$replace = $2 . '|' . $1;

# Now that you know what you want to replace, search and replace.
$line =~ s/$search/$replace/gs;

print $line;


Let me know if this is helpful
exit;

 

[BStopp]

I understand what you posted, but unfortunately it doesn't address my problem. The actual script won't have any knowledge prior to run-time of what pattern to match, nor what the new value will be. Therefore, there is no way to know just how many $1, $2..$n variables will be used. All of this information is contained within a property file.

If i were to code in the $1, $2 replacement values, it would work, but i need it to work by specifying the information in a flat file.

My co-worker and i basically boiled the problem down:

The script reads the properties file correctly.
It pattern matches the files against the first column correctly.

When it gets to the replacement, it treats the value contained within the $changes{$newname} variable as a litteral string, where as I need it to treat it as I see it, and that's a string with variables that need to be interpolated. This is what isn't happening, 1 interpolation happens, but i need 2 instances of it to occur.

Hopefully that clarifies some things.. doubt it though

-B

 

[KevinADC]

Can you show how you are building the hash you are using for the substitution, that might be where the problem is.

 

[mikedaruke]

I understand you. Try this:

If you need to do a second level of variable interpolation in the replacement pattern, you can do this:

s/(\\$\w+)/\$1/eeg;



#Variables
$line = 'abckdefgh123456';
$search = '(...).*?h(1.*?6)';
$replace = '$1 . \'-\' . $2' ;
# Now that you know what you want to replace, search and replace.
$line =~ s/$search/$replace/ee;

print $line;

 

[BStopp]

Sure, I'll show you how I'm building the hash, if it'll help. Here goes:

my $INAME = 0;
my $ONAME = 1;
my %changes = ();
my @line;
my $key;
my $old_file;
my $new_file;
my $dirh = new DirHandle;
my @files;

open(FILE, "<change.defs");
while (<FILE>) {
    @line = split(/\s+/, $_);
    $key = shift(@line);
    $changes{$key} = push(@line);
}
close FILE;

opendir($dirh, "$ARGV[0]")
@files = grep {!/^\./} readdir($dirh);
closedir $dirh;

foreach (@files) {
   $old_file = $_;
   $new_file = $_;
   foreach $key (sort keys %changes) {
      if ($old_file =~ /$changes{$key}[$INAME]/) {
              $new_file =~ s/$changes{$key}[$INAME]/$changes{$key}[$ONAME]/g;
              print $new_file . "\n";
              exit;
      }
   }
}

where the file "changes.defs" contains the following:

KEY1    B_(Stopp)_(\d{4})(\d{2})\d{2}   B$1_Y$2_M$3

Now I only have it printing the '$new_file' variable out to the screen to validate that it works. But every time i run it, i get back the following litteral value:

B$1_Y$2_M$3

The directory that's passed as an argument has the file named: B_Stopp_20060324 in it.... so when printing $new_file, it should print: BStopp_Y2004_M03

But as mentioned it doesn't..

My coworker was here and he says "What we need to do is have Perl treat a string that is read from a file as if it was code."

Any new thoughts on the matter?

-B - really appreciating the assistance.

 

[KevinADC]

hmm... this line is probably not doing what you want:

$changes{$key} = push(@line);

Maybe this is what you mean to do:

push @{$changes{$key}},[@line];


and you will need to use something like mikedaruke posted so the variables will be interpolated from the input file, which as you can see treats the lines as single-quoted strings and will not interpolate or expand variables.

 

[BStopp]

Sorry all, there was a mistake in my code when i tried to recreate the functionality (my actual code can't be posted). Here is code the recreates my problem and i don't know how to fix:

use DirHandle;

my $INAME = 0;
my $ONAME = 1;
my %changes = ();
my @line;
my $key;
my $old_file;
my $new_file;
my $dirh = new DirHandle;
my @files;

open(FILE, "<change.defs");
while (<FILE>) {
    @line = split(/\s+/, $_);
    $key = shift(@line);
    push(@{$changes{$key}}, @line);
}
close FILE;

opendir($dirh, "$ARGV[0]");
@files = grep {!/^\./} readdir($dirh);
closedir $dirh;

foreach (@files) {
   $old_file = $_;
   $new_file = $_;


   foreach $key (sort keys %changes) {

      if ($old_file =~ /$changes{$key}[$INAME]/) {
            $new_file =~ s/$changes{$key}[$INAME]/$changes{$key}[$ONAME]/g;
            print $new_file . "\n";
            exit;
      }
   }
}

 

[BStopp]

I tried changing the regex substitution statement to be s///eeg, and s///ee, like mikedaruke suggested, and got this error:

Bareword found where operator expected at (eval 1) line 1, near "$1_Y"
        (Missing operator before _Y?)
Bareword found where operator expected at (eval 1) line 1, near "$2_M"
        (Missing operator before _M?)

-B

 

[BStopp]

Before anyone suggests it, i also tried changing the "change.defs" file to read:

B${1}_Y${2}_M${3} and B($1)_Y($2)_M($3)

thinking that it was the _ that was causing the problem. Still recieved the same error.

-B

 

[mikedaruke]

BStopp,

Put your replace in the 'qq[]' and it will work.

$replace = 'qq[B$1_Y$2_M$3]';

Let me know how you make out.

 

[KevinADC]

this is working for me:

foreach $key (sort keys %changes) {
   if ($old_file =~ /$changes{$key}[$INAME]/) {
      $new_file =~ s/$changes{$key}[$INAME]/interpolate(0,$1,$2,$3,$changes{$key}[$ONAME])/e;
      print "$new_file\n";
   }
}
sub interpolate {
   my @args = @_;
   my $pattern = pop(@args);
   for my $i (1..$#args) {
      $pattern =~ s/\$$i/$args[$i]/g;
   }
   return($pattern);
}

but it's not very elegant, maybe someone else will know a better way to get B$1_Y$2_M$3 to interpolate.

 

[KevinADC]

actually, this is a bit better:

foreach $key (sort keys %changes) {
   if ($old_file =~ /$changes{$key}[$INAME]/) {
      my @matches = $old_file =~ /$changes{$key}[$INAME]/g;
      $new_file =~ s/$changes{$key}[$INAME]/interpolate($changes{$key}[$ONAME],@matches)/e;
      print "$new_file\n";
   }
}
sub interpolate {
   my ($pattern,@args) = @_;
   for my $i (0..$#args) {
      $pattern =~ s/\$@{[$i+1]}/$args[$i]/g;
   }
   return($pattern);
}

 

[BStopp]

You guys are great! I used the sub-routine that KevinADC provided and it does exactly what I need to do.

There's only one thing i don't get about the code though. On the left side of the substitution you have:

@{[$i+1]}

I get that if it's not exactly like this it doesn't work, but what i don't undersand is why that is, what exactly does this syntax do for the script?

-B

 

[KevinADC]

the matching strings are numbered $1 $2 $3 etc, but the array holding the results of the matches starts with 0 so we need to add one to it to make $1 and $array[0] correspond to each other. The syntax I used is just a tricky way to add one to $i without using a another variable to accomplish the same thing, like this:

sub interpolate {
   my ($pattern,@args) = @_;
   [b]my $n = 0;[/b]
   for my $i (0..$#args) {
      [b]$n++;[/b]
      $pattern =~ s/\$[b]$n[/b]/$args[$i]/g;
   }
   return($pattern);
}

I picked that trick up a while back after reading "Programming Perl":

Here's a trick for interpolating the value of a subroutine call into a string:

print "My sub returned @{[ mysub(1,2,3) ]} that time.\n";

It works like this. At compile time, when the @{...} is seen within the double-quoted string, it's parsed as a block that will return a reference. Within the block, there are square brackets that will create a reference to an anonymous array from whatever is in the brackets. So at run-time, mysub(1,2,3) is called, and the results are loaded into an anonymous array, a reference to which is then returned within the block. That array reference is then immediately dereferenced by the surrounding @{...}, and the array value is interpolated into the double-quoted string just as an ordinary array would be. This chicanery is also useful for arbitrary expressions, such as:

print "That yields @{[ $n + 5 ]} widgets\n";

Be careful though. The inside of the square brackets is supplying a list context to its expression. In this case it doesn't matter, although it's possible that the above call to mysub() might care. When it does matter, a similar trick can be done with a scalar reference. It just isn't quite as pretty:

print "That yields ${ \($n + 5) } widgets.";