Tar shell scripting problems

[ajrandrup]

hello, im a newbie in unix scripting. can someone pls send me a sample script that will tar files in the folders of a specific directory, but will exclude specific files, and afterwards, will ftp the tar into another server.

for example:
in this directory, /export/home/acrandru/testing/

the following files and folders will be tar'd (this are not all the files/folders in this directory)

copyright.html
license.txt
license.html
classes --this folder also has files inside--
src --this is a folder

but will exclude these files in these directories:

export/home/acrandru/testing/copyright.html
export/home/acrandru/testing/license.txt

I have tried to create a script located in /export/home/acrandru/ays/tar_files.sh

this is a sample of the file
#!/bin/ksh

# Uncomment "set -x" below for script debugging
set -x

# Set Environment Variables ----- BEGIN

# The set -a option exports all variables
# until the set +a command is executed
set -a

. /opt/psoft/.profile 1

LOGS_DIR=/export/home/acrandru
SCRIPTS_DIR=/export/home/acrandru
OUT_DIR=/export/home/acrandru/
SCRIPT_NAME=tar_files.sh
SCRIPT_LOG=$SCRIPT_NAME.log

set +a

# Set Environment Variables ----- END

# Show env. variable int text file for debugging
set > $LOGS_DIR/$SCRIPT_LOG

cd $OUT_DIR
echo $OUT_DIR >> $LOGS_DIR/$SCRIPT_LOG
path=`pwd`;
echo $path >> $LOGS_DIR/$SCRIPT_LOG
CUR_MON=`date|awk '{print $2}'`;
CUR_YR=`date +%Y`;
echo "CURRENT MONTH: " $CUR_MON >> $LOGS_DIR/$SCRIPT_LOG
echo "CURRENT YEAR: " $CUR_YR >> $LOGS_DIR/$SCRIPT_LOG

if [ $path = $OUT_DIR] ;then
(
echo "Tar monitoring directory" >> $LOGS_DIR/$SCRIPT_LOG
$ tar cf /export/home/acrandru/tar.tar --exclude-from=/export/home/acrandru/ays/exlude_files.txt *
echo "Creating tar file for aonprod08" >> $LOGS_DIR/$SCRIPT_LOG
ExitCode=0

if [ $ExitCode -ne 0 ]
then
echo "Cannot create tar file" >> $LOGS_DIR/$SCRIPT_LOG
fi
)
fi

exit 0

_____

when i try to run the script, i get an error:

CUR_MON=Dec
+ + date +%Y
CUR_YR=2003
+ echo CURRENT MONTH: Dec
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ echo CURRENT YEAR: 2003
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ echo Tar monitoring directory
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ $ tar -cf /export/home/acrandru/tar.tar --exclude-from=/export/home/acrandru/ays/exclude_files.txt exclude_files.txt tar_files.sh tar_files.sh.log tar_files2.sh tar_files3.sh tar_files3.sh.log
./tar_files3.sh[55]: $: not found


If anyone out there can help me out, i'd really appreciate it.

You can email me at ajrandrup@yahoo.com, or if you would like to discuss with me, my aim id is aysrandrup, or at yahoo, ajrandrup

 

[jad]

create a file called '/export/home/acrandru/ays/include_files.txt' that has:

copyright.html
license.txt
license.html
classes
src

and then:

cd /export/home/acrandru/
tar cf /export/home/acrandru/tar.tar -I /export/home/acrandru/ays/include_files.txt

 

[PHV]

What is the meaning of the $ before the tar command ?

Hope This Help
PH.

 

[ajrandrup]

hello again, before I tried using your helpful reply, i got this msg after I ran the script
CUR_MON=Dec
+ + date +%Y
CUR_YR=2003
+ echo CURRENT MONTH: Dec
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ echo CURRENT YEAR: 2003
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ echo Tar monitoring directory
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ cd /export/home/acrandru/
+ tar cf /export/home/acrandru/tar.tar --exclude-from=/export/home/acrandru/exclude_files.txt ays exclude_files.txt local.cshrc local.login local.profile tar.tar testing
tar: can't change directories to --exclude-from=/export/home/acrandru: No such file or directory
tar: tar.tar same as archive file
+ echo Creating tar file for aonprod08
+ 1>> /export/home/acrandru/ays//tar_files3.sh.log
+ ExitCode=0
+ [ 0 -ne 0 ]
+ exit 0


but when i checked /export/home/acrandru, i have created a tar.tar file:

acrandru psoft 50257408 Dec 10 23:18 tar.tar

how do i open this file? and the error I got: cant change directories..blah,blah...does this mean that although I have tar'd the folder, i wasnt able to exclude the files I wasnt supposed to tar?

Thanks!

 

[jad]

try typing:

tar tf tar.tar

in the /export/home/acrandru directory and you will see the 'contents' of the tar archive.

in your exclude file if you have a directory name it will exclude all files in that directory, if you don't include the directory, but include all files in the directory the directory will still be archived.

 

[ajrandrup]

Hi again. I've already iincluded the full path of the files to be excluded in my exclude_files.txt:

export/home/acrandru/testing/humana.parm
export/home/acrandru/testing/dma.parm
export/home/acrandru/testing/definity.parm
export/home/acrandru/testing/aon.parm
export/home/acrandru/exclude_files.txt
export/home/acrandru/ays/tar_files.sh
export/home/acrandru/ays/tar_files2.sh
export/home/acrandru/ays/tar_files.sh.log
export/home/acrandru/ays/tar_files3.sh.log
export/home/acrandru/ays/tar_files3.sh
export/home/acrandru/tar.tar

Here is my directory structure;

acrandru:/export/home/acrandru>ls -lrt

local.profile
local.login
local.cshrc
testing
tar.tar
exclude_files.txt
ays
tar_filesworking.sh
tar_files3.sh

I cant still seem to exclude the specific files.Thanks, by the way for the tip on how to view the content of the tar file

 

[ajrandrup]

Hello, I've decided to do the include_files.txt instead, and it works! Now, I've been given an additional problem. I have to ftp the tar.tar file to another machine!
I've tried adding this to the script

`ftp -ndv machine.name.com $HOME/tar_files3.sh.log >> $LOGS_DIR/$SCRIPT_LOG
user acrandru password
put tar2.tar
bye
EOT`

when i run the script, after doing the tar procedure, my machine sort of hangs, and when i check the log file, i get:

CURRENT MONTH: Dec
CURRENT YEAR: 2003
Tar monitoring directory
machine.nam.com: bad port number-- /export/home/acrandru/tar_files3.sh.log
usage: -ndv host-name [port]
ftp>

** should i be using the machine.name.com, or the exact port number??(actually i've used the port number, 123.456.789, and I got the same message)

 

[jad]

does:

ftp -ndv machine.name.com <<EOT >> $HOME/tar_files3.sh.log 
       user acrandru password
       put tar2.tar
       bye
EOT

work?

 

[ajrandrup]

thanks! yes, it does work! is there any way that i can put the machine name, user name and password into file that can be called by the script? this is for security, so that if the script is viewed,the login data is not seen.

also, i would like the log file to show the directory and file names of the files being put in the tar. as of now, the only content of my log file is the following:

Archiving on 121403
Tar monitoring directory
Archiving on 121403 (this even repeats!)
Tar monitoring directory
Creating tar file for aonprod08
__________________________________
right now, my script looks like this:
_______________________________________
#This will tar all files in aonprod08/818sp9_80

Date=`date +"%m%d%y"`
cd $HOME
OUT_DIR=/export/home/acrandru/
echo "Archiving on $Date" >> $HOME/tar_files2_$Date.sh.log

echo "Tar monitoring directory" >> $HOME/tar_files2_$Date.sh.log
cd $OUT_DIR
# for file in `ls -lrt |awk '{print $3}`
# do
# echo $HOME $file >> $HOME/tar_files2_$Date.sh.log
tar cfX $OUT_DIR/arc__818sp9_80_$Date.tar exclude_tar.lis *
echo "Creating tar file for aonprod08" >> $HOME/tar_files2_$Date.sh.log
# done


`ftp -ndv atldev9 <<EOT>> $HOME/tar_files2_$Date.sh.log
user acrandru password
put arc__818sp9_80_$Date.tar
bye
EOT`

exit 0

i'm pretty sure, writing for file in find... or something like that will do the job of listing out the filenames in the log file, im just not sure of the syntax.

thanks a lot!

 

[jad]

well you could try:

#This will tar all files in aonprod08/818sp9_80

Date=`date +"%m%d%y"`
cd $HOME
OUT_DIR=/export/home/acrandru/
echo "Archiving on $Date" >> tar_files2_$Date.sh.log

    echo "Tar monitoring directory" >> tar_files2_$Date.sh.log
    cd $OUT_DIR
    tar cfX $OUT_DIR/arc__818sp9_80_$Date.tar exclude_tar.lis *
     echo "Creating tar file for aonprod08" >> tar_files2_$Date.sh.log
     tar tf arc__818sp9_80_$Date.tar >> tar_files2_$Date.sh.log

echo "Enter FTP Username: \c" ; read USERNAME
echo "Enter FTP Password: \c" ; read PASSWORD

ftp -ndv atldev9 <<EOT>> tar_files2_$Date.sh.log
       user $USERNAME $PASSWORD
       put arc__818sp9_80_$Date.tar
       bye
EOT

exit 0

 

[ajrandrup]

hello, thanks again. doing the tar tf command does put the files into the logfile.

however, i'm not quite sure i undertand the part abt the user name and password. what i would like to do is to create a separate file that contains the user name and password,for example:
ftp.pass would contain
username
password

and this file will be called by the script as a variable or something like that. although i did try doing your suggestion, it seems that the script &quot;goes out&quot; to get the name and password, but doesnt input the name and password.

thank you very much!

 

[PHV]

Try something like this:

#echo "Enter ftp Username: \c" ; read USERNAME
#echo "Enter ftp Password: \c" ; read PASSWORD
user=`cat ftp.pass`
ftp -ndv atldev9 <<EOT>> tar_files2_$Date.sh.log
       user $user
       put arc__818sp9_80_$Date.tar
       bye
EOT

Hope This Help
PH.

 

[ajrandrup]

thanks very much!

 

[ajrandrup]

hello! i swear this would be my last posting on this subject. As you can read above, I have to tar some files, then ftp it to another machine. I've already done that with much help from this forum.
Now I have a new problem, and that is, after I ftp the tar file to the other server I now have to untar (open it). By the way, we've already changed the tar file to be tar.gz, that would be the file to be ftp'd to the machine, and then will have to be untar'd/ungzipped.

 

[jad]

look into using ssh, you can then pass commands and run an untar script on the other machine.
alternatively if you are moving between machines on the same network it might well be possible to do this via either mounting the disk (rather than ftp) and pipe to a different file system, or you could use the not as secure rsh in the pipe and untar the file in a network stream.