Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations TouchToneTommy on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Help to reduce the processing time of a program 1

Status
Not open for further replies.

rklad77

Technical User
Oct 2, 2003
5
US

I have got a program to extract data from 100 files and write it to 1 master file and 3
slave files. Each file in 100 input file contains 12000 rows and 4 columns. Data is to be
extracted for every entry in the first column of master file. The program takes ages to complete as
everytime it has to open a file and read the contents from start.
Can anybody please help to get around this by using better tool..
I am pasting the program...

++++++++++++++++++++++++++++++++++++++++++++++++++
real tim
real xmov(101)
real ymov(101)
real zmov(101)
integer majcnt
integer sidr
real sfa
real sfo
real offa
real offo
integer dattyp
character carry*1
integer dof
integer vad
integer lcid
real sf
integer vid
integer death
integer birth
integer nid
integer lineadd
integer nend
integer nserch
integer nofil
integer fstnod(1000)
integer cnt
integer tally
integer node(100000)
real x(101)
real y(101)
real z(101)
character nodeserch*5
integer serch
integer serchmas
integer serchslv
character filetoconv*25
character ansa*1
character ansb*1
character filenam(101)*32
character dummy*4
call system("clear")
call system("rm X_Curves.key Y_Curves.key Z_Curves.key")
call system("rm Boundary_Cards.key")
call system("clear")
print*,'Please ensure *NODE is on the same line number'
print*,'in all your files !'
print*,' '
print*,'Please enter line number of *NODE keyword'
read*,nserch
print*,' '
print*,'Please enter line number of last node'
read*,nend
print*,' '
print*,'Would you like to list .key files prior to file'
print*,'selection ?'
read(*,'(a)')ansa
if(ansa.eq.'y') then
call system("ls *.key")
end if
print*,' '
print*,'Enter first 25 character filename to be read'
print*,'(this should read filename00.key)'
read(*,'(a)')filetoconv
print*,' '
print*,'How many key files do you have (1-100)'
read*,nofil
print*,' '
* print*,'Your DYNA files will be Boundary_Cards.key'
* print*,'and X/Y/Z_Curves.key. If you have files of'
* print*,'these names please delete them first!'
* print*,' '
* print*,'Enter a key to carry on ..'
* read*,carry
*
*
* establsih file names
*
*
tally = 1
do cnt = 0,9
filenam(tally)=filetoconv//char(cnt+48)//char(48)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(49)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(50)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(51)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(52)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(53)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(54)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(55)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(56)//'.key'
tally = tally + 1
filenam(tally)=filetoconv//char(cnt+48)//char(57)//'.key'
tally = tally + 1
end do
*
*
* write boundary cards ..
*
*
vad=2
sf=1.0
vid=0
death=0
birth=0
lcid=1
open(1,status='old',file=filenam(1))
open(2,status='new',file="Boundary_Cards.key")
write(2,*)'*KEYWORD'
do serchmas = 1,nserch
read(1,'(a5)')nodesearch
end do
do serchslv = 1,nend-nserch
read(1,880)node(serchslv)
dof=1
write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
dof=2
lcid=lcid+1
write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
dof=3
lcid=lcid+1
write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
lcid=lcid+1
end do
write(2,*)'*END'
close(1)
close(2)
*
* write curve cards ..
*
*
tim=0.001
lcid=1
sidr=0
sfa=1.0
sfo=1.0
offa=0.0
offo=0.0
dattyp=0
linadd=0
open(2,status='new',file="X_Curves.key")
open(3,status='new',file="Y_Curves.key")
open(4,status='new',file="Z_Curves.key")
write(2,*)'*KEYWORD'
write(3,*)'*KEYWORD'
write(4,*)'*KEYWORD'
do majcnt = 1,nend-nserch
do serch = 1,nofil
open(1,status='old',file=filenam(serch))
do serchmas = 1,nserch+lineadd
read(1,'(a5)')nodesearch
end do
read(1,900)node(serch),x(serch),y(serch),z(serch)
* print*,node(serch),x(serch),y(serch),z(serch)
close(1)
end do
do serch = 1,nofil
if (serch.eq.1) then
goto 10
end if
xmov(serch)=x(serch)-x(serch-1)
ymov(serch)=y(serch)-y(serch-1)
zmov(serch)=z(serch)-z(serch-1)
* print*,xmov(serch)
* print*,ymov(serch)
* print*,zmov(serch)
write(2,910)tim,xmov(serch)
write(3,910)tim,ymov(serch)
write(4,910)tim,zmov(serch)
tim=tim+0.001
goto 20
*
10 write(2,*)'*DEFINE_CURVE'
write(2,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
write(2,910)0,0
lcid=lcid+1
write(3,*)'*DEFINE_CURVE'
write(3,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
write(3,910)0,0
lcid=lcid+1
write(4,*)'*DEFINE_CURVE'
write(4,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
write(4,910)0,0
lcid=lcid+1
*
20 end do
lineadd=lineadd+1
tim=0.001
end do
write(2,*)'*END'
write(3,*)'*END'
write(4,*)'*END'
close(1)
close(2)
close(3)
*
*
880 format(i8)
890 format(i10,i10,i10,i10,f10.1,i10,i10,i10)
900 format(i8,f16.6,f16.6,f16.6)
910 format(f20.4,f20.4)
920 format(i10,i10,f10.1,f10.1,f10.1,f10.1,i10)
*
*
end
 
The only speed improvement I can see is in removing the if statement from the loop.
Code:
10      write(2,*)'*DEFINE_CURVE'
        write(2,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(2,910)0,0
        lcid=lcid+1
        write(3,*)'*DEFINE_CURVE'
        write(3,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(3,910)0,0
        lcid=lcid+1
        write(4,*)'*DEFINE_CURVE'
        write(4,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(4,910)0,0
        lcid=lcid+1        
*
      do serch = 2,nofil
        xmov(serch)=x(serch)-x(serch-1)
        ymov(serch)=y(serch)-y(serch-1)
        zmov(serch)=z(serch)-z(serch-1)
*        print*,xmov(serch)
*        print*,ymov(serch)
*        print*,zmov(serch)
        write(2,910)tim,xmov(serch)
        write(3,910)tim,ymov(serch)
        write(4,910)tim,zmov(serch)
        tim=tim+0.001
20    end do
You will get a minor improvement if nofil is big. No improvement if nofil is small. I don't know if rewriting it in another language will improve the speed. You could also try the various optimization switches on the compiler but for an I/O bound program it may not make very much difference.
 
Since the program is apparently IO bound, One idea.. not for the faint of heart .. is to read the file in much larger 'chunks'. Suppose something like 1000 bytes at a time, c/r and l/f included (read it as a binary).

Then parse the data strings.. and read more data as needed. In some, cases the improvement is astounding..in others .. not worth the effort.

You could try a small test: part a read the file record by record; part b read the chunks. See if ou see any major change.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top