Efficient use of grep to set a variable 1

[Frank2]

Hi,

I (newbie shell scripter) have to write a script that will read parameter/value pairs used as configuration parameters for an application. The configuration file as the following form:

parameter1Name=parameter1Value
parameter2Name=parameter2Value
...

I'm looking for a more efficient way of reading through the configuration file in order to set script variables. I don't like the fact that I'm using the same grep twice...

if grep "^parameter1Name=" file.txt > /dev/null; then
  # parameter1Name= is found in file.txt
  aVar=`grep "^FOO=" file.txt | awk -F= '{ print $2 }'`
else
  # FOO= is not found in file.txt
  aVar="UNDEFINED"
fi

Many thanks!

 

[gamerland]

Frank -- So we do not waste your time with an incorrect response, would you please provide a little more detail concerning what you want to do with the parameters once you load them?

- Can they be processed serially and then diregarded or do we need to keep track of all at once?
- Will they be used in the script or are you planning to export them for use outside the script?
- Does the file contain anything except parameters that needs to be filtered out first, like comments?
- You are grepping for a parmname, do you need to know the name or can we presume the next one matches?

This looks like a real straight forward case for a while/read loop provided we can eliminate the extraneous records.

-Mike.

 

[Frank2]

First, thanks for the help Mike.

Here is an example of the config file (which we will refer to as config.txt) I have to "parse":

0,1,15,360,string1,string2,...,string n
PARAM1=VALUE1
PARAM2=VALUE2

As you can see the configuration file format mixes CSV with PARAM=VALUE format. Why? Good question, I just inherited the support of a 10000 lines script calling other scripts and the headache that comes with it

Anyhow the goal is to assign each value found in the config file to variables that will be used in the script. Here's how it's actually done:

firt_line=`head -1 config.txt `
scriptVar1=`echo $firt_line | awk -F, '{ print $1 }' `
scriptVar2=`echo $firt_line | awk -F, '{ print $2 }' `
scriptVar3=`echo $firt_line | awk -F, '{ print $3 }' `
scriptVar4=`echo $firt_line | awk -F, '{ print $4 }' `
scriptVar5=`echo $firt_line | awk -F, '{ print $5 }' `
scriptVar6=`echo $firt_line | awk  -F, '{ for (i=6; i<NF+1; i++) print $i }' `
scriptVar7=`awk '/PARAM1/ { print }' config.txt | awk -F= '{ print $2 }'`
if grep &quot;^PARAM2=&quot; config.txt > /dev/null; then
  # PARAM2 is found in the config.txt
  scriptVar8=`grep &quot;^PARAM2=&quot; config.txt | awk -F= '{ print $2 }'`
else
  # PARAM2 is not found in config.txt
  scriptVar8=&quot;UNDEFINED&quot;
fi

# So in the end scriptVars would have the following values:
# scriptVar1=0
# scriptVar2=1
# scriptVar3=15
# scriptVar4=360
# scriptVar5=string1
# scriptVar6=string2,...string n
# scriptVar7=VALUE1
# scriptVar8=VALUE2 (or UNDEFINED if PARAM2 is not found in config.txt)

The above code works just fine but I don't like the fact that I'm using the same "grep" twice to set scriptVar8.

To answer your questions:
- The scriptVars needs to be accessible by other (child) scripts
- The config file format is always as found above
- The parameter names are fixed and known (except in some cases PARAM2 does not exist).

Many thanks again!

/Frank

 

[CaKiwi]

This is untested but may do what you want.

scriptvar8=`awk -F= '/^Param2/{print $2;exit}END{print "UNDEF"}' config.txt`

CaKiwi

"I love mankind, it's people I can't stand" - Linus Van Pelt

 

[gamerland]

Frank - I rarely question why files contain what they do when I inherit code. Just leave it as a mystery of the ages until you know enough about the process to clean it up.

To simply eliminate the second grep, just save your answer from the first and check it. This script goes through grep a second time, because the first told you something exists, and the second now assigns it.

So, replace this:
if grep "^PARAM2=" config.txt > /dev/null; then
# PARAM2 is found in the config.txt
scriptVar8=`grep "^PARAM2=" config.txt | awk -F= '{ print $2 }'`
else
# PARAM2 is not found in config.txt
scriptVar8="UNDEFINED"
fi
#
#** with **
#
P2INPUT=`grep "^PARAM2=" config.txt | awk -F= '{ print $2 }'`
if [ -n "$P2INPUT" ]
then
scriptVar8=$P2INPUT
else
scriptVar8="UNDEFINED"
fi

Read as: If P2INPUT is not null (or more accurately longer than zero) assign it to Var8, else it is empty...

What you see here is one of the most common things I run into with grep. People ask the question, then after getting the answer, use basically the same process to assign it.

Also, I hope it was a typo and that you do not really have a single 10,000 line script. If you do, my condolences. ;-)

 

[tdatgod]

Hi,
I think the first question in the orignal post was how to avoid reading the file more than once.

I think that means they want something like

exec 4< Paramfile
while read line <&4
do
a=`expr $a + 1`
if ( $a == 1 ); then
stuff from above
else if ( $a == 2 ); then

.
.
.
done

 

[Frank2]

Thank you all for your great help!

And Mike, no it was not a typo... it's really a 10,000 lines script!!

/Frank

 

[Ygor]

The most efficient way is for awk to parse the config file and pipe the result through read.

#!/usr/bin/ksh
awk 'BEGIN {e=s=&quot;UNDEF&quot;}
NR==1 {gsub(&quot;,&quot;,&quot; &quot;o=$0}
/^PARAM1=/ {split($0,a,&quot;=&quot;s=a[2]}
/^PARAM2=/ {split($0,a,&quot;=&quot;e=a[2]}
END {print s,e,o}' config.txt |read scriptVar7 scriptVar8 scriptVar1 scriptVar2 scriptVar3 scriptVar4 scriptVar5 scriptVar6

echo scriptVar1=$scriptVar1
echo scriptVar2=$scriptVar2
echo scriptVar3=$scriptVar3
echo scriptVar4=$scriptVar4
echo scriptVar5=$scriptVar5
echo scriptVar6=$scriptVar6
echo scriptVar7=$scriptVar7
echo scriptVar8=$scriptVar8

scriptVar1=0
scriptVar2=1
scriptVar3=15
scriptVar4=360
scriptVar5=string1
scriptVar6=string2 ... string n
scriptVar7=VALUE1
scriptVar8=VALUE2

 

[gamerland]

Ygor -- can you give us the English version of what your awk statement is saying? I like it, I just want to ensure I know what it is saying.

Thanks,
-Mike