Background control 1

[kwillg]

I have a cron job that scp/ssh to hosts in an array @host and does a task on the array based on the args passed. i.e. dothis -hlouie,suzie,craig,irene -d (d is the task it's doing and the array contains louie, suzie, craig and irene). I don't want them to be kicked off one at a time and wait for the task to finish. I need to know how to get a rolling queue of 2/3 tasks running. Then when the task returns, pop the next entry from the bottom of the array. I know how to do the pop, and the iteration through the array to call the task. I need help with the (child) job control and I have no experience with fork.

 

[justice41]

Can you post the relevant part of what you have tried so far? It would be easier to offer suggestions and ideas with some code to look at and twiddle.

jaa

 

[kwillg]

I am running the cron task as user locuser; passing -n option to decifer which task to run; and passing -h option with args: louie,suzie,craig,irene.

0 5 * * * /usr/xxx/cron.runtask -hlouie,suzie,craig,irene -n
within cron.runtask
setup environment
./runtask $* ##($* = -hlouie,suzie,craig,irene -d)

Within runtask, the taskN piece works one at a time. I don't want to background 40 scp instances to run simultaneously. So I did a grep for the scp's I kicked off.

if ($opts{'n'}) {
while (@host) {
$lasthost = pop(@host);
print LOG "TaskN on $lasthost \n";
TaskN ($lasthost);
}
}

sub CheckProcs {
$blah = `ps -ef | grep scp | grep locuser |wc -l`;
print LOG "$blah = # of scp's by locuser.\n";
return $blah;
}


sub TaskN {
my $user = "me";
my $dist = "/usr1/some/directory";
my $rem_dir = "/usr/remote/directory";
@filelist = `ls $dist`;

foreach $host (@_) {
open (LOG, ">>/home/kwillg/temp/logs/$0.log&quot || die "can't open $0.log";
foreach $file (@filelist) {
chomp $file;
chomp($timestamp = `date '+%h %d %T'`);
print LOG "$timestamp $0 $$ pushing $file -> $host \n";
scp ("$dist/$file", "$user\@$host\:$rem_dir"
}
}
close LOG;
}

 

[justice41]

Test this out. It replaces the [tt]if ($opts{'n})[/tt] statement. It appears to run ok for some dummy tasks that I made up.

my @queued = ();
my $maxqueue = 2; # number of concurrent procs allowed in queue

if ($opts{'n'}) {

       while (@host or @queued) {

            print "Host: @host\nQueue: @queued\n";
            my $lasthost = pop(@host) if @host;
            if ( defined($lasthost) and my $pid = fork ) { # parent process

                push @queued, $pid;

            } elsif (defined $pid) {            # child process

                print LOG "TaskN on $lasthost \n";
                TaskN ($lasthost);
                exit;

            }
            next if @host and @queued < $maxqueue;
            while (1) {

                my $nextpid = shift @queued;
                last if waitpid($nextpid, WNOHANG);
                push @queued, $nextpid;
                sleep 1;

            }
       }
}

jaa

 

[justice41]

Oops, you should take out the following line

print "Host: @host\nQueue: @queued\n";

I just put it there for debugging.

jaa

 

[justice41]

I also forgot to mention that you need to use the POSIX module to get the [tt]WNOHANG[/tt] flag for the [tt]waitpid[/tt] command.

use POSIX ':sys_wait_h';

jaa

 

[kwillg]

Hey Justice, works great. The only thing I had to change was what I expected as input to the subroutine TaskN. I passed it an array of hosts before whereas now I'm passing it one host at a time. I assumed that since it was being called TaskN($lasthost) that I could just read the value as $_ within the subroutine. After about 30 mins. of confusion I got it to work using @_. I'm still don't quite get that piece but I'll read up some more and see if it clicks. Thanks for the help Justice.

 

[MikeLacey]

@_ is the name of the array that contains all the values passed to your subroutine, ie.

($first_val, $second_val, $third_val) = @_; Mike
________________________________________________________________

"Experience is the comb that Nature gives us, after we are bald."

Is that a haiku?
I never could get the hang
of writing those things.