conditional sed statement 1

[wtrepani]

I have a file with over 5M lines in it. I want to use sed to remove any line where x > 91. In order to get X, I have some logic to run, therefore piping it to sed rewrites 5M + lines for every line it processes. I remember seeing a way of running a conditional sed (sed '/s'/'if etc'/') but can't remember the format. Any out there have any ideas???

Thanks

Will

 

[PHV]

What about man sed ?
Another tip: man awk

Hope This Help
PH.

 

[wtrepani]

Tried both. I am using awk to mine the specific data I am testing in the command. I want to use that inside of sed. man sed does not go into condition statements.

Something like - sed '/'`read each line, |parse.pl | awk NF-1| if X > 91 `'/d'. That way it will delete every line where the next to last line is >91, however, it has to run through parse.pl to get this number.... Otherwise it would be easy...

 

[Chacalinc]

could you post a line where X > 91 and X < 92? what is the logic to get the X value?

 

[wtrepani]

We take one line from the file, run it through a perl script, awk - to $(NF-1) to get the value.

In a script we would do;
while true
do
A=`head -1 file`
B=`head -1 file |perl /path/script.pl |awk ' { print $(NF-1) ] '`
if [ B -gt 90 ]
do
sed '/'$A'/d' filein > fileout
done
fi
done

However, by doing this you are reading a file with 5M+ lines once for every line. I am trying to find a way I can run sed once (thus reading the file once) and putting a condition to read the line, parse it with the pl script, awk it, check the value, and whack it if the condition is true.

I was thinking about something along the lines of

sed '/'` A=1 B=0 C=`head -$A file |tail -$B` D=`head -$A file |tail -$B |perl script.pl |awk ' { print $(NF-1) } ' if [ $D -gt 90 ] do echo $C done let A=A+1 let B=B+1 `'/d' filein > fileout

But it hasn't yet.....

 

[PHV]

If your perl script isn't too complex, I guess all can be done with only one awk script, or even with only one perl script.
Can you explain what the perl script is supposed to do ?

 

[wtrepani]

takes output from fwtmp and converts to days.

output from fwtmp:

XXXUSER tc pts/tc 7550 8 0000 0000 1058653784 Jul 19 17:29:44 2003

Output from perl:

XXXUSER tc pts/tc 7550 8 0000 0000 1058653784 Jul 19 17:29:44 2003 204 days

Then I awk for $(NF-1), --> 204, if greater than X, whack it.

 

[Chacalinc]

Ok, I don't know what your perl script does, but all you need is &quot;getline&quot; function of awk and &quot;cat&quot; your file, an example to give you a hint:

cat your_file | awk '{ getline; if ((NF-1)<92) {print $0}}'

this line command will display only lines where X is < than 92.

Hope this help...

 

[Chacalinc]

sorry:

cat your_file | awk '{ getline; if ($(NF-1)<92) {print $0}}'

 

[Chacalinc]

ok.. I didn't read you last post, sorry. complete solution:

cat your_file | awk '{ getline; print $0}' | perl script.pl | awk ' { if ($(NF-1)>91) { print $0 >> file_out  } }

 

[PHV]

Why not just pipe the output of fwtmp to a perl script (based on the one you already have) that outputs only records within the 90 days. You'll avoid a lot of unnecessary head/tail stuff.

Hope This Help
PH.

 

[wtrepani]

Chacalinc,

Almost had it. The only problem now is when I look at the output, it is the output of the perl script, not the fwtmp command.

ie

`/usr/sbin/acct/fwtmp -ci < /var/adm/wtmp > /tmp/wtmp.out`

`cat /tmp/wtmp.out |awk ' { getline; print $0 } ' |perl script.pl |awk ' { if ($(NF-1)<91) { print $0 } } ' >> /tmp/wtmp.new`

then `/usr/sbin/fwtmp -ic < /tmp/wtmp.new > /var/adm/wtmp` will not create the /var/adm/wtmp file correctly. I need to verify the ($(NF-1)<91) then output the original line to the new file, with all the original spaces. That's why I originally was going to delete uneeded lines from a file rather than rewrite it as output. The format of the /var/adm/wtmp file must be maintained for proper logging operation (ie. the date must always start at position 57).

It is pretty well guarenteed that when I find a line that doesn't get deleted nothing after it will be deleted either. Is it possible to read the entire file into buffer then delete the first line over and over from the buffer but not write it to disk until ($(NF-1)<91)?

PHV, I'll try that in the morning. I'm to tired at this point to do it right....

Thanks all for your assistance.

Will

 

[Chacalinc]

could you modify your perl script in order to return 0 if X<92 ans 1 if X is >91? or return days may be...

so you can have:

cat /tmp/wtmp.out |awk ' { A=$0;
                         B=system("perl script.pl " $0); 
                         if (B=0) { print A >> "wtmp.out" }
                         } '

Cheers.