sed - delete blank lines between regular expression

[gotchausa]

How do I delete blank lines from lines containing certain wors/characters using sed? E.g:

Hello there
a

b
c
Bye bye

How can I delete the blank line between lines a and b?

 

[olded]

I think you're asking to delete blank lines:

sed '/^$/d' whateveryourfilenamesis

Regards,

Ed

 

[gotchausa]

Actually, I want to delete only blank lines between 2 specific lines, not all blank lines from a file.

 

[olded]

npv:

OK, now I understand. awk probably isn't the best tool for this, but here it is:

#!/bin/ksh

# I use nawk; you may not
echo ""|nawk ' BEGIN { start="a"; stop="b"; cnter=0; startflag=0; stopflag=0;
while ((getline < &quot;x.file&quot > 0)
{ cnter++; arr[cnter] = $0 }
}
{

for (i=1; i<=cnter; i++)
{
if(arr == start)
{
startflag++
print arr
continue
}

if(startflag && cnter > (i+1))
{
x=i+1
if(arr[x] == stop && arr ~ /^$/)
{
startflag=0
continue
}

}
print arr
}

} '

I have to look ahead so I read all the data into an array. I echo &quot;&quot; to the awk script because there is no input file so I force one iteration.

I search and keep printing until I find start, a and print that. Check to see if the present element is blank and if the next element is the stop. If it is continue and don't print the blank.

I check to make sure I'm not overstepping the array boundary.

Bigoldbulldog: can you do something better with a sed script?

Regards,

Ed

 

[vgersh99]

Why wouldn't this work?

nawk -f rmBlanks.awk textFile.txt

#----------- rmBlanks.awk --------

/^[a]/,/^/ { if ($0 ~ /^$/ ) next; print; next }
1
#----------------------------------

 

[olded]

vgersh:

I like yours better than mine. You're using a range pattern, it looks like. I need to read up on that.

Looks like the browse also cratered by [ and ] so I'm reposting.

Regards,

Ed

#!/bin/ksh

echo &quot;&quot;|nawk ' BEGIN { start=&quot;a&quot;; stop=&quot;b&quot;; cnter=0; startflag=0; stopflag=0;
while ((getline < &quot;x.file&quot > 0)
{ cnter++; arr[cnter] = $0 }
}
{

for (i=1; i<=cnter; i++)
{
if(arr == start)
{
startflag++
print arr # needs to be print arr
continue
}

if(startflag && cnter > (i+1))
{
x=i+1
if(arr[x] == stop && arr ~ /^$/)
{
startflag=0
continue
}

}
print arr # needs to be print arr
}

 

[bigoldbulldog]

Here are two sed solutions

#! /usr/bin/sed -nf
/^a/{
:loop
N
s/.*\n//
/^b/q
p
b loop
}

#! /usr/bin/sed -nf
/^a/,/^b/{
/^a/b
/^b/{
q
}
p
}

Don't forget the -n. vgersh's solution didn't work on my version so I got it working with these changes. I put in the exit in case the file is really huge. I assume the sought range to print is done one time.

#! /usr/bin/awk -f
/^a/,/^b/ {
if( $0 !~ /^a/ && $0 !~ /^b/ ) print
else if($0 ~ /^b/) exit
}

Both sed solutions perform about the same (negligible difference) if the sought range is short but the second is about 30% faster if the range to print is huge. They are about 4 to 5 times faster then the awk solution.

P.S. only the first solution allows the start expression &quot;^a&quot; to be printed if it appears again before
the stop expression &quot;^b&quot;. mpv, you need to decide if this is what you want or even if you want the shortest range using the latest possible start string.


Cheers,
ND

bigoldbulldog@hotmail.com

 

[vgersh99]

Seems like 'old' Solaris '/usr/bin/awk' is having syntax problems with my implementation, but either nawk or /usr/xpg4/bin/awk [POSIX compliant awk] are fine. I tend to stay away from the 'old' Solaris awk anyway....

Nice sed implementations, ND - I really like the terseness and the performance of 'basic' sed. Thanks for keeping up with the alternate implementations.

vlad

 

[gotchausa]

Thank you all for your suggestions, I'll try each one and will post the successful ones.

 

[gotchausa]

OK, I think I need to post the actual file that needs editing, here it is:

FMember: SDKLF
IPAddress: 2.2.2.2

FMember: NND
IPAddress: 47.1.2.3

FMember: NODE112
IPAddress: 47.100.11.12

FMember: NODE11
IPAddress: 47.11.12.16

FMember: NODE1
IPAddress: 47.102.11.22
NodeID: 100


FGroup: COMM
Member: NODE112

Member: NODE1


FGroup: TELE
Member: NND


FGroup: ABTR
Member: NODE11
Member: NODE100

Member: NODE232

Member: NODE342
Member: NODE122

Each FGroup contains a number of Members. What I need to do is to delete ALL and ONLY blank lines between the lines containing 'Member: NODExxx' for each FGroup. I dont want to delete lines that separate the individual FGroup sections(each FGroup is usually separated by 3 blank lines but this is not always the case).

 

[CaKiwi]

Does this awk script do what you want?

{
   if (NF == $0) {
     j++
   }
   else {
     if ($1 == "FGroup:") {
       for (k=0; k<j; k++) print &quot;&quot;
     }
     print
     j = 0
   }
}

Hope this helps. CaKiwi

 

[gotchausa]

Thanks. Tried it, didnt do anything to my original file

 

[bigoldbulldog]

Try out this sed script. If it's also no good then send us a copy in input and the expected output.

#! /usr/bin/sed -f
/Member: NODE/{
:loop
$!N
/\n$/{
b loop
}
/\nMember: NODE/{
P
s/.*\n//
b loop
}
}

Cheers,
ND

bigoldbulldog@hotmail.com

 

[gotchausa]

No, the sed script didnt work - keeps looping and doesnt actually exit.
Here's the desired output with the previously posted input file.

FMember: SDKLF
IPAddress: 2.2.2.2

FMember: NND
IPAddress: 47.1.2.3

FMember: NODE112
IPAddress: 47.100.11.12

FMember: NODE11
IPAddress: 47.11.12.16

FMember: NODE1
IPAddress: 47.102.11.22
NodeID: 100


FGroup: COMM
Member: NODE112
Member: NODE1


FGroup: TELE
Member: NND


FGroup: ABTR
Member: NODE11
Member: NODE100
Member: NODE232
Member: NODE342
Member: NODE122

 

[CaKiwi]

My script seems to do what you want with a small modification.

{
   if (NF == $0) {
     j++
   }
   else {
     if ($1 == "FGroup:" || $1 == "FMember:") {
       for (k=0; k<j; k++) print &quot;&quot;
     }
     print
     j = 0
   }
}

Run it by entering

awk -f my-script your-input > your-output

CaKiwi

 

[Einstein47]

Maybe I am oversimplifying here, but it looks like the easiest solution would be to remove all the empty lines, and then add an extra newline before each of the lines whose first character is an 'F'.

Something like:

sed '/^$/d' input.txt | awk '/^F/ {print ""} {print $0}' > output.txt

When 900 years old you are, this good look. Einstein47
(Love is like PI - natural, irrational, endless, and very important.)

 

[bigoldbulldog]

I couldn't find any problems with the sed solution I posted. If anyone can figure out its failure on your machine then I am curious too.

Here's a quick awk solution and if anyone has better I am eager to see it.

#! /usr/bin/awk -f
BEGIN {
keeper = &quot;Member: NODE&quot;
blank = &quot;^$&quot;
}
{
if( $0 ~ keeper ){
inrange = &quot;T&quot;
print
ct=0
}
else if( $0 ~ blank && inrange == &quot;T&quot; ){
ct++
}
else{
for(;ct > 0; --ct) printf(&quot;\n&quot
print
inrange = &quot;F&quot;
}
}

Cheers,
ND

bigoldbulldog@hotmail.com

 

[gotchausa]

OK, Einstein47's solution worked, CaKiwi's didnt change the original file.

Einstein47, is it possible to add 2 xtra newlines instead of one?

Thank you all for your help!

 

[vgersh99]

From the top of my head [might have to optimize a bit]:

nawk -f rmBlanks.awk originalFile.txt

#------------------ rmBlanks.awk ---------------

BEGIN {
fm=0;
}

/^Member:/ { fm=1 ; print ; next }
fm == 1 && /^$/ { arr=arr &quot;\n&quot;; next }
/^FGroup:/ { printf(&quot;%s&quot;, arr); fm=0; arr=&quot;&quot; }
1

#--------------------------------------------------

vlad

 

[Einstein47]

Sure, just change the print &quot;&quot; to print &quot;\n\n&quot;. In fact you can put anything you want in there as a separator to make it more readable.

Good luck. Einstein47
(Love is like PI - natural, irrational, endless, and very important.)