Files, Databases, and character matches OH MY! 1

[Masher1]

Ok, I am fairly new at Unix scripting, but have been a programmer for years. I have a project I am spinning my wheels on right now.

Here is the job, I have a directory full of files. There are 5 types of files in this directory, all of them are tab delimited text files for a database. they follow a format: fileType_date_location.

I have made a stored procedure for each file to load the data from the file into the db.

What I need to do is use a script to basically get a list of all the files in the directory, then go file by file, determine what kind of file it is, trim the top 2 lines off the file, and then call the correct stored procedure (via command line) based off of the file type. After all the files have been looped through I need to clear out the directory.

I have tried making a variable with the number of files in the directory (ls | wc -l) and then loop through each file .. but im not really sure how to progress. If anyone has any ideas or some approaches to think about that would be much appreciated.

 

[Masher1]

To clarify I am basically looking for some kind of logic like:

For each file in directory(
if type 1 then
run comannd(filename)

if type 2 then
run comannd(filename)

if type 3 then
run comannd(filename)

if type 4 then
run comannd(filename)

if type 5 then
run comannd(filename)

done



well, thats the basic gist at least.

 

[Masher1]

well, im close now.

this is what I have for now.

for inFile in $*
do
if [$infile = "type1*"] <--this does not work and is the sticking point atm--
then
calldb command
remove file
elif [.. = "type2*"]
...so on for all the file types
done

so what I need to do now is find some way for my input (string that is the filename) to eval to true on a partial match (say the first few chars) to the file type. I tried this regular expression match using a wild card, and that bombed. Im guessing its becuase its in quotes and being seen as a literal. Could I use some kind of concat to make it work. Or could I use a tool like grep inside the if to see if there is the text im looking for and have that eval to true on a match?

Issue I have atm

 

[SamBones]

Try a [tt]case[/tt] statement instead of a series of [tt]if-the-elif[/tt]'s.

for inFile in $*
do
  case $inFile in
    type1*)    # do something to type1
               # calldb command
               # remove file
               ;;
    type2*)    # do something else
               ;;
    *)         print "Unknown file type!"
               ;;
  esac
done

 

[Masher1]

LOL. Cute how the case lets me do the match, and the if goes bonkers. Oh well, that got me through that =) Thanks!

Next lil glitch. I need to trim the top two lines off the file before I send it to my import procedure.

In my case I have
type1*)
tail +3 $inFile < $inFile+temp <--- this does not work.
dbinsert procedure($inFiletemp)
rm $inFile
rm $inFiletemp

The idea (which works fine at the command line) was to tail the file, drop the first two line, dump the output into a temp file, then load the temp file, then clean up the mess.

I assumed a standard concat would work (+) to give me my filenameTemp. Turns out shell scripting hates me and that wont work. So, how to I use my file name (held in inFile) plus some end tag to house the cleaned up data from tail?

I feel like a confused kid out here. Drives me nuts, as in normal programming this stuff is so easy! =P

 

[Masher1]

Ya know what. I think I am just being stubborn. I think I can just use a static name and then rm at the end.

 

[Masher1]

Grrr. Well I tried

tail +3 $inFile > type1_temp

Result

No such file or directory

So I tried

touch touch type1_temp
tail +3 $inFile > type1_temp

Result

It shows the records on standard out (my screen) as they should look after the tail command, and makes the file type1_temp. The file is empty though =/

 

[Masher1]

ugg .. chmod 755 on my folder and it works.

 

[SamBones]

Dude! Calm down! You're posting things here at a rate of 3.5 posts per reply! Take a deep breath! Relax!

Did you solve your problem? Or are you stuck on something. It's hard to tell with your multiple posts.

Hang in there, all will be well.

 

[SamBones]

The [tt]if[/tt] does work. Try it like this...

#!/bin/ksh

for inFile in $*
do
    if [[ ${inFile} = @(type1*) ]]
    then
        # do something to type1
    elif [[ "$inFile" = "type2*" ]]
        # do something else
    elif [[ $inFile = type3* ]]
        # do another something else
    fi
done

The double square brackets in the Korn shell are much more forgiving.

To concatenate "temp" to your file name...

tail +3 $inFile > ${inFile}temp
dbinsert procedure(${inFile}temp)
rm ${inFile}
rm ${inFile}temp

The reason the "[tt]+temp[/tt]" didn't work is because it was thinking that the plus sign was part of the variable name. Just taking off the plus will confuse it more, it will think your variable name is "[tt]inFiletemp[/tt]" and will fail. To make it easier for the shell to parse your variable name correctly, use braces in the form of "[tt]${VAR}[/tt]".

Hope this helps.