Scripting Exercises Needed!!! 2

[COTLG]

Hi All,

I am a unix admin, am familiar with unix and want to be proficient in scripting.

I love to take up challenges and surmount them hence I need a place/site where I can get a lot of scripting (preferably korn shell) exercises to solve.

I also do not mind being given an exercise by any individual on or outside this forum (as long as it is not someone's school work). I will do it and send to the person.

I will appreciate any contribution to my request.

Thanks.

 

[SamBones]

I find helping people in this forum hones my skills. I've learned a lot just by helping people.

 

[motoslide]

I learn a lot just by reading along.

Here's a sample problem:

I can produce a text file showing the execution times of a program (line-by-line). I want to review the results with the desire to find where there might be performance issues. Luckily, the TRACE includes execution times following the line of code like this:

0330 FIND (3)X1$,X2$,X3,X4
    {715.45}{8.52}
0335 PRINT (4)"Hello World"
    {934.31}{218.85}
0340 FIND (5)A0$
    {1152.90}{21.59}

In the above example, line 335 took 218mS to execute (the second value). I want to review thousands of lines in this file and only display the ones where the execution time is greater than some threshold (say 100mS).

There are examples of doing this kind of "Display the line BEFORE a patterm match" in these forums, but I'm having problems because the execute times are surrounded by curly brackets.

I don't have an immediate need for a solution, but it's one that just popped up again and I had just read your post asking for a challenge.


"Proof that there is intelligent life in Oregon. Well, Life anyway.

 

[COTLG]

Hi Motoslide,

Here is the answer to your question. Perl script handles file manipulation excellently.

$variable2 will have to be modified a bit to remove "{". I will do that later.

textfile2 contains your text output.

----------------------------------------------

#!/usr/bin/perl

$file = "textfile2";
open(DAT, $file) || die("File could not be opened");
@array1 = <DAT>;
close (DAT);

foreach $line (@array1)
{
chop $line;
($variable1,$variable2) = split(/\}/, $line);
print $variable2;
print "\n";
}

------------------------------------------------

Best regards,
Chike.

 

[COTLG]

Hi Motoslide,

Here is a complete answer to your question:

------------------------
#!/usr/bin/perl

$file = "textfile2";
open(DAT, $file) || die("File could not be opened");
@array1 = <DAT>;
close (DAT);

foreach $line (@array1)
{
chomp $line;
($variable1,$variable2) = split(/\}/, $line);
$stringlength = length($variable2);
$newvariable = substr($variable2,1,$stringlength-1);
if ($newvariable > 50)
{
print $newvariable;
print "\n";
}
}
------------------------------------

I hope it meets your expectations?

Best regards,
COTLG.

 

[motoslide]

Chike,

Thanks for the work.

I get a list of those "process times" which exceed 50, but it doesn't print the line which is associated with that time (the preceding line in this case).

Example output:

0335 PRINT (4)"Hello World"
{934.31}{218.85}

or

0335 PRINT (4)"Hello World"
218.85

So, to walk through the process, I need to read the text file, find a line where the number in the second set of curly brackets is greater that 50 (or some other threshold), then print that line and the previous line.

For more scripting fun, you might want to check Forum 1551 (Puzzles for Programmers).



"Proof that there is intelligent life in Oregon. Well, Life anyway.

 

[COTLG]

Thanks Motoslide.

 

[mrn]

What about this one

thread822-1232376

Mike

"A foolproof method for sculpting an elephant: first, get a huge block of marble, then you chip away everything that doesn't look like an elephant."

http://www.airport-parking-site.co.uk/

 

[KenCunningham]

Hi. Forum1551 might be of some interest to you too.

 

[PHV]

motoslide, an awk way:
awk -v v=50 '
/}$/{
x=substr($NF,1,length($NF)-1);sub(/.*{/,"",x)
if(x+0>v)print a"\n"$0
}
{a=$0}
' /path/to/input

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[motoslide]

PHV,

I don't think my shell liked your awk:

awk: error compiling regular expression /.*{/: *?+ not preceded by valid expression
source line 3 of program <<
/}$/{
x=substr($N ... >>
context is
>>> x=substr($NF,1,length($NF)-1);sub(/.*{/,"",x) <<<

I had to escape the "{" character in the sub field:
old
sub(/.*{/,"",x)
new
sub(/.*\{/,"",x)

It works great, but I don't understand where it's preserving the previous line.

Thanks PHV for the shell script solution, and COTLG for the Perl method. I really should learn Perl someday.







"Proof that there is intelligent life in Oregon. Well, Life anyway.

 

[PHV]

but I don't understand where it's preserving the previous line
{a=$0}

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[COTLG]

This also answers your question completely Motoslide:

----------------
!/usr/bin/perl

$file = "textfile2";
open(DAT, $file) || die("File could not be opened");
@array1 = <DAT>;
close (DAT);

foreach $line (@array1)
{
chomp $line;
($variable1,$variable2) = split(/\}/, $line);
$stringlength = length($variable2);
$newvariable = substr($variable2,1,$stringlength-1);
if ($newvariable > 50)
{
print $formerline;
print "\n";
print $newvariable;
print "\n";
}
else
{
$formerline = $line;
!/usr/bin/perl

$file = "textfile2";
open(DAT, $file) || die("File could not be opened");
@array1 = <DAT>;
close (DAT);

foreach $line (@array1)
{
chomp $line;
($variable1,$variable2) = split(/\}/, $line);
$stringlength = length($variable2);
$newvariable = substr($variable2,1,$stringlength-1);
if ($newvariable > 50)
{
print $formerline;
print "\n";
print $newvariable;
print "\n";
}
else
{
$formerline = $line;
}
}
----------------------------

Thanks.

 

[COTLG]

Sorry for the mixup....

------------------------------------------
#!/usr/bin/perl

$file = "textfile2";
open(DAT, $file) || die("File could not be opened");
@array1 = <DAT>;
close (DAT);

foreach $line (@array1)
{
chomp $line;
($variable1,$variable2) = split(/\}/, $line);
$stringlength = length($variable2);
$newvariable = substr($variable2,1,$stringlength-1);
if ($newvariable > 50)
{
print $formerline;
print "\n";
print $newvariable;
print "\n";
}
else
{
$formerline = $line;
}
}
----------------------------
Thanks.

 

[motoslide]

Both solutions work great!
Stars all around.
Time to greet the weekend.

Just for comparison, I ran both scripts on a 500000 line text file. Perl won by a narrow margin, but each took less than 3 seconds to complete.

"Proof that there is intelligent life in Oregon. Well, Life anyway.

 

[pattbo]

OK, I am a network guy with minimal scripting experience... I need to parse a firewall log file to see what rules are or are not in use. I have a list of rule numbers I want to search for the occurence of in the log file /var/log/messages ... I was thinking I could grep for "rule number" in file "/var/log/messages ... and count the occurences. This will take a lot of time as I have 500+ rule numbers to search for and this file will be 100's of MB a day... . I need to figure out the logic to look for occurence of a rule number and then log to file if that "rule number" is in use, and also the list of rules numbers not in use.

being able to read in a file, formatted however with each of the rules in it, will also make life easier...

Can anyone provide help with this task?

Thanks in Advance...

Pat

 

[pattbo]

Sorry but need to update my request... darn firewall won't let me export rule numbers... here is the sample date... in csv format.. 134102 is the number I need to check if it exists, and then check the next one.. Anyone want to tackle this problem? I assume that the 13th field is always the rule number...

Jun 5 16:14:45 10.20.0.252 "2006-06-05 16:01:35","690928575","10.20.0.34","Packet filter","Notification","New connection","Allow","UDP (User Datagram Protocol)","172.23.92.148","172.17.52.40","1094","389","134102",,,,,,"NIC #2",,,,,,,,,,,,,,"Den1-corpabgw2",,,

 

[rimss]

How about this one ..

thread822-1239537

http://www.tek-tips.com/viewthread.cfm?qid=1239537&page=1

Thanks,