Help with SED, creating script from file list 3

MCubitt · Jan 21, 2005

I have generated a file list which contains files (not directories) with the full path preceeding them, eg:
/oracledata/EDI/control01.ctl
/oracledata/EDI/EDI.dbf
/oracleindex/EDI/control02.ctl
/redolog1/EDI/redo01.log
/oracle/app/oracle/oradata/EDI/system01.dbf
/oracle/app/oracle/oradata/EDI/temp01.dbf

From that list i would like to create two new files, but deleting any line which has temp*.dbf and arranging the format, so...
(/backuploc is in a variable
TOBEDECIDED is in a variable)
file1:
cp -p /oracledata/EDI/control01.ctl /backuploc/oracledata/EDI/control01.ctl
cp -p /oracledata/EDI/EDI.dbf /backuploc/oracledata/EDI/EDI.dbf
cp -p /oracleindex/EDI/control02.ctl /backuploc/oracleindex/EDI/control02.ctl
cp -p /redolog1/EDI/redo01.log /backuploc/redolog1/EDI/redo01.log
cp -p /oracle/app/oracle/oradata/EDI/system01.dbf /backuploc/oracle/app/oracle/oradata/EDI/system01.dbf

file2:
cp -p /backuploc/oracledata/EDI/control01.ctl /oracledata/TOBEDECIDED/control01.ctl
cp -p /backuploc/oracledata/EDI/EDI.dbf /oracledata/TOBEDECIDED/TOBEDECIDED.dbf
cp -p /backuploc/oracleindex/EDI/control02.ctl /oracleindex/TOBEDECIDED/control02.ctl
cp -p /backuploc/redolog1/EDI/redo01.log /redolog1/TOBEDECIDED/redo01.log
cp -p /backuploc/oracle/app/oracle/oradata/EDI/system01.dbf /oracle/app/oracle/oradata/TOBEDECIDED/system01.dbf

Later, when a restore script is run I'd like to change "TOBEDECIDED" to be a selected value, so another find/replace.

Applications Support
UK

MCubitt · Jan 24, 2005

Future,

The script is very close now, but this happens:
cp -p /ifsdoc/Backup/oracle/IFSD/initIFSD.ora /oracle/TOBEDECIDED/initIFSD.ora

this should be
cp -p /ifsdoc/Backup/oracle/IFSD/initIFSD.ora /oracle/TOBEDECIDED/initTOBEDECIDED.ora

(the file name should change too)

thanks

Applications Support
UK

MCubitt · Jan 24, 2005

In fact, it really only SHOULD do a substitution on the second occurence of the path & file, so
srcdb=IFSD
desdb=TOBEDECIDED

/oracle/IFSD/test/initIFSD.ora

refers to
/oracle/TOBEDECIDED/test/initTOBEDECIDED.ora

to give on the restore
cp -p /oracle/backup/IFSD/test/initIFSD.ora /oracle/TOBEDECIDED/test/initTOBEDECIDED.ora

Applications Support
UK

MCubitt · Jan 24, 2005

Sorry for my audible ramblings, I think I managed to work it out myself..

From my main script:

Code:

awk -f bandr.unix backuploc=$despath rscript=$restorescript olddb=$srcdb newdb=$desdb $filelist>$backupscript

bandr.unix:

Code:

BEGIN {
FS = OFS = "/"
}

# Skip lines with "temp*.dbf".
toupper($0) ~ /TEMP[^\/]*\.DBF$/ { next }

{
backup = backuploc $0
# Make new folder (if not present)
printf "mkdir -p $(dirname %s) 2> /dev/null\n",
backup
# Copy file
printf "cp -p %s %s\n", $0, backup
DB = $(NF-1)
# destination is to be decided
$(NF-1) = newdb
sub( /olddb/,newdb,$NF )
printf "cp -p %s %s\n", backup, $0 >rscript
}

Thanks for your help and inspiration!

Applications Support
UK

MCubitt · Jan 24, 2005

Actually, it only did this for the last line... and corrupted the other bits.

Clearly a case of me misunderstanding the command!

.
.
.
FINE:
cp -p /ifsdoc/Backup/BackupofIFSD/redolog2/IFSD/undotbs01.dbf /redolog2/DESTINATIONDBTOBEDECIDED/undotbs01.dbf

WRONG:
cp -p /ifsdoc/Backup/BackupofIFSD/oracle/app/oracle/product/9.2.0.1.0/dbs/initIFSD.ora /oracle/app/oracle/product/9.2.0.1.0/DESTINATIONDBTOBEDECIDED/initIFSD.ora

WRONG:
cp -p /ifsdoc/Backup/BackupofIFSD/oracle/app/oracle/admin/IFSD/pfile/initIFSD.ora /oracle/app/oracle/admin/IFSD/DESTINATIONDBTOBEDECIDED/initIFSD.ora

WRONG:
cp -p /ifsdoc/Backup/BackupofIFSD/oracle/app/oracle/product/9.2.0.1.0/dbs/lkIFSD /oracle/app/oracle/product/9.2.0.1.0/DESTINATIONDBTOBEDECIDED/lkIFSD

WRONG:
cp -p /ifsdoc/Backup/BackupofIFSD/home/oracle/keymed/Backup/controlDESTINATIONDBTOBEDECIDED.sql /home/oracle/keymed/DESTINATIONDBTOBEDECIDED/controlDESTINATIONDBTOBEDECIDED.sql

Applications Support
UK

futurelet · Jan 24, 2005

Run with
awk -f backup.awk backuploc="/foo" rscript="file2" data >file1

Code:

# FS tells Awk where to split the line it read ($0)
# into the fields $1, $2, ... , $NF.  $NF is the last field.
# OFS is the ouput field-separator; when assigning
# to a field, OFS is put between the fields to
# rebuild $0.
BEGIN {
  FS = OFS = "/"
}

# Skip lines with "temp*.dbf".
toupper($0) ~ /TEMP[^\/]*\.DBF$/ { next }

{
  backup = backuploc $0
  printf "mkdir -p $(dirname %s) 2> /dev/null\n",
    backup
  printf "cp -p %s %s\n", $0, backup
  DB = $(NF-1)
  $(NF-1) = "TOBEDECIDED"
  sub( "^" DB, "TOBEDECIDED", $NF )
  printf "cp -p %s %s\n", backup, $0  >rscript
}

When the input file is

/oracledata/EDI/control01.ctl
/oracledata/EDI/EDI.dbf
/oracleindex/EDI/control02.ctl
/redolog1/EDI/redo01.log
/oracle/app/oracle/oradata/EDI/system01.dbf
/oracle/app/oracle/oradata/EDI/temp01.dbf

output file1 is

mkdir -p $(dirname /foo/oracledata/EDI/control01.ctl) 2> /dev/null
cp -p /oracledata/EDI/control01.ctl /foo/oracledata/EDI/control01.ctl
mkdir -p $(dirname /foo/oracledata/EDI/EDI.dbf) 2> /dev/null
cp -p /oracledata/EDI/EDI.dbf /foo/oracledata/EDI/EDI.dbf
mkdir -p $(dirname /foo/oracleindex/EDI/control02.ctl) 2> /dev/null
cp -p /oracleindex/EDI/control02.ctl /foo/oracleindex/EDI/control02.ctl
mkdir -p $(dirname /foo/redolog1/EDI/redo01.log) 2> /dev/null
cp -p /redolog1/EDI/redo01.log /foo/redolog1/EDI/redo01.log
mkdir -p $(dirname /foo/oracle/app/oracle/oradata/EDI/system01.dbf) 2> /dev/null
cp -p /oracle/app/oracle/oradata/EDI/system01.dbf /foo/oracle/app/oracle/oradata/EDI/system01.dbf

and output file2 is

cp -p /foo/oracledata/EDI/control01.ctl /oracledata/TOBEDECIDED/control01.ctl
cp -p /foo/oracledata/EDI/EDI.dbf /oracledata/TOBEDECIDED/TOBEDECIDED.dbf
cp -p /foo/oracleindex/EDI/control02.ctl /oracleindex/TOBEDECIDED/control02.ctl
cp -p /foo/redolog1/EDI/redo01.log /redolog1/TOBEDECIDED/redo01.log
cp -p /foo/oracle/app/oracle/oradata/EDI/system01.dbf /oracle/app/oracle/oradata/TOBEDECIDED/system01.dbf

Let me know whether or not this helps.

If you have nawk, use it instead of awk because on some systems awk is very old and lacks many useful features. Under Solaris, use /usr/xpg4/bin/awk.

For an introduction to Awk, see faq271-5564.

MCubitt · Jan 24, 2005

Thanks future, but I almost have worked it out..

It now looks:

Code:

BEGIN {
FS = OFS = "/"
}

# Skip lines with "temp*.dbf".
toupper($0) ~ /TEMP[^\/]*\.DBF$/ { next }

{
backup = backuploc $0
# Make new folder (if not present)
printf "mkdir -p $(dirname %s) 2> /dev/null\n",backup
# Copy file
printf "cp -p %s %s\n", $0, backup
sub(/olddb/,newdb)
printf "cp -p %s %s\n", backup, $0 >rscript
}

called by
awk -f bandr.unix backuploc=$despath rscript=$restorescript olddb=$srcdb newdb=$desdb $filelist>$backupscript

The only problem is, I cannot seem to get awk to see olddb as a variable not constant. if I add $ it does "nothing".

Applications Support
UK

vgersh99 · Jan 24, 2005

sub(olddb,newdb)

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

MCubitt · Jan 24, 2005

Thank you Vlad, was driving me nuts. No matter where I looked, nothing seemed to tell me that!

Now:

Code:

# awk code to create backup and restore scripts
# from file list.

BEGIN {
FS = OFS = "/"
}
# Skip lines with "temp*.dbf".
toupper($0) ~ /TEMP[^\/]*\.DBF$/ { next }
{
backup = backuploc $0
# Make new folder (if not present)
printf "mkdir -p $(dirname %s) 2> /dev/null\n",backup
# Copy file
printf "cp -p %s %s\n", $0, backup
# destination is to be decided
gsub(olddb,newdb)
printf "cp -p %s %s\n", backup, $0 >rscript
}

I presume I can drop the FS bit as I need no file separator?

Applications Support
UK

futurelet · Jan 24, 2005

As Vlad said, change

sub(/olddb/,newdb)

to

sub(olddb,newdb)

Your program requires "olddb=$srcdb" on the command line.
Isn't olddb simply the next to last field of a line from the input file? So why should you have to pass that value to your program?

MCubitt · Jan 24, 2005

Future,
I have olddb in the command line:
awk -f bandr.unix backuploc=$despath rscript=$restorescript olddb=$srcdb newdb=$desdb $filelist>$backupscript

olddb can vary where it appears in the input file and may appear multiple times in a line. The full input file (notice the latter lines):

Code:

/oracledata/IFSD/control01.ctl
/oracledata/IFSD/control03.ctl
/oracledata/IFSD/ifsapp_archive_data01.dbf
/oracledata/IFSD/ifsapp_data01.dbf
/oracledata/IFSD/ifsapp_data02.dbf
/oracledata/IFSD/ifsapp_data03.dbf
/oracledata/IFSD/ifsapp_data04.dbf
/oracledata/IFSD/ifsapp_data05.dbf
/oracledata/IFSD/ifsapp_data06.dbf
/oracledata/IFSD/ifsapp_data07.dbf
/oracledata/IFSD/ifsapp_data08.dbf
/oracledata/IFSD/ifsapp_data09.dbf
/oracledata/IFSD/ifsapp_data10.dbf
/oracledata/IFSD/ifsapp_data11.dbf
/oracledata/IFSD/ifsapp_report_data01.dbf
/oracledata/IFSD/oem_repository01.dbf
/oracledata/IFSD/temp01.dbf
/oracleindex/IFSD/control02.ctl
/oracleindex/IFSD/ifsapp_archive_index01.dbf
/oracleindex/IFSD/ifsapp_index01.dbf
/oracleindex/IFSD/ifsapp_index02.dbf
/oracleindex/IFSD/ifsapp_index03.dbf
/oracleindex/IFSD/ifsapp_index04.dbf
/oracleindex/IFSD/ifsapp_index05.dbf
/oracleindex/IFSD/ifsapp_report_index01.dbf
/oracleindex/IFSD/users01.dbf
/redolog1/IFSD/redo01.log
/redolog1/IFSD/redo02.log
/redolog1/IFSD/redo03.log
/redolog1/IFSD/redo04.log
/redolog1/IFSD/redo05.log
/redolog1/IFSD/system01.dbf
/redolog2/IFSD/redo01a.log
/redolog2/IFSD/redo02a.log
/redolog2/IFSD/redo03a.log
/redolog2/IFSD/redo04a.log
/redolog2/IFSD/redo05a.log
/redolog2/IFSD/undotbs01.dbf
/oracle/app/oracle/product/9.2.0.1.0/dbs/initIFSD.ora
/oracle/app/oracle/admin/IFSD/pfile/initIFSD.ora
/oracle/app/oracle/product/9.2.0.1.0/dbs/lkIFSD
/home/oracle/keymed/Backup/controlIFSD.sql

Applications Support
UK

MCubitt · Jan 24, 2005

In the above example, IFSD is the value of olddb, in case you had not guessed.

thanks.

Applications Support
UK

vgersh99 · Jan 24, 2005

olddb can vary where it appears in the input file and may appear multiple times in a line.

in this case, shouldn't
sub(olddb,newdb)

be

gsub(olddb,newdb)

?

also, be careful passing variables to awk - quote your variable assignments in case they contain blanks:

awk -f bandr.unix backuploc="$despath" rscript="$restorescript" olddb="$srcdb" newdb="$desdb" $filelist>$backupscript

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

MCubitt · Jan 24, 2005

vlad, thanks for the tip.. oh, and I changed it to gsub (see previous).

Thanks all!

Applications Support
UK

futurelet · Jan 24, 2005

MCubitt said:
awk and sed are both equally awful, powerful, but disgustingly unclear!

I hope you've changed your mind. Complex regular expressions are easier to write than to read (in any programming language), but Awk is less ugly than Perl.

MCubitt · Jan 24, 2005

I'll take your word for it, Ive had only limited misfortune to have to script Unix.

cheers

Applications Support
UK

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Help with SED, creating script from file list 3

MCubitt

Programmer

MCubitt

Programmer

MCubitt

Programmer

MCubitt

Programmer

MCubitt

Programmer

futurelet

Programmer

MCubitt

Programmer

vgersh99

Programmer

MCubitt

Programmer

futurelet

Programmer

MCubitt

Programmer

MCubitt

Programmer

vgersh99

Programmer

MCubitt

Programmer

futurelet

Programmer

MCubitt

Programmer

Similar threads

Part and Inventory Search

Sponsor