memory problem 1

[momphy]

HI guys

when i run my fortran code . the memory get less and less and finely run out . i'm not family with memory work .here is my code :

program rfcor

character*32,dimension(20,160)::synfile
character*32 syn,synname,obspath,filepath
character*64 outpath
character*32,dimension(3329):bsfile
character*32,dimension(300)::filename
* real,dimension(3500)::syndata,obsdata
real,allocatable,dimension)::syndata,obsdata
integer ni,nj,filenum,num
integer,dimension(20,160)::corr





syn='syndata/data.txt'
open(300,file=syn)
do i=1,20
do j=1,160
read(300,*,end=600)synfile(i,j)
end do
end do
600 continue

open(400,file='list.txt')
read(400,*)obsfile
*******************************************************
*up read station name
********************************************************
do ni=1,3329
obspath=trim(obsfile(ni))//'/data.txt'
* print*,obspath

open(401,file=obspath)
num=0
filename=''
do while(.true.)
num=num+1
read(401,*,end=500)filename(num)
end do
500 continue
print*,num
filenum=num-1
* print*,filenum
******************************************************
*read event name
******************************************************
do nj=1,filenum
filepath=trim(obsfile(ni))//'/'//trim(filename(nj))
print*,filepath
allocate(obsdata(3500))
obsdata=''
print*,obsdata
open(402,file=filepath)
read(402,*)(obsdata(nk),nk=1,3500)
* print*,size(obsdata)

program rfcor

character*32,dimension(20,160)::synfile
character*32 syn,synname,obspath,filepath
character*64 outpath
character*32,dimension(3329):bsfile
character*32,dimension(300)::filename
* real,dimension(3500)::syndata,obsdata
real,allocatable,dimension)::syndata,obsdata
integer ni,nj,filenum,num
integer,dimension(20,160)::corr





syn='syndata/data.txt'
open(300,file=syn)
do i=1,20
do j=1,160
read(300,*,end=600)synfile(i,j)
end do
end do
600 continue

open(400,file='list.txt')
read(400,*)obsfile
*******************************************************
*up read station name
********************************************************
do ni=1,3329
obspath=trim(obsfile(ni))//'/data.txt'
* print*,obspath

open(401,file=obspath)
num=0
filename=''
do while(.true.)
num=num+1
read(401,*,end=500)filename(num)
end do
500 continue
print*,num
filenum=num-1
* print*,filenum
******************************************************
*read event name
******************************************************
do nj=1,filenum
filepath=trim(obsfile(ni))//'/'//trim(filename(nj))
print*,filepath
allocate(obsdata(3500))
obsdata=''
print*,obsdata
open(402,file=filepath)
read(402,*)(obsdata(nk),nk=1,3500)
* print*,size(obsdata)
***********************************************************
*read syn data
************************************************************
do i =1,20
do j=1,160
synname='syndata/'//synfile(i,j)
* print*,synname
allocate(syndata(3500))
syndata=''
open(301,file=synname)
read(301,*)(syndata(k),k=1,3500)
* print*,'syndata'
* print*,size(syndata)
corr(i,j)=sum(obsdata*syndata)
deallocate(syndata)
end do
end do
deallocate(obsdata)
************************************************************
*write result in to txt file
***********************************************************
outpath=filepath//'rfcor.txt'
print*,outpath
open(700,file=outpath,action='write')
do i=1,20
write(700,*)(corr(i,j),j=1,160)
end do
end do
end do
end program rfcor
i only give one variable and overwrite in loop . but this take more and more memory . anybody know about this problem
thaks a lot

 

[ArkM]

Please, look at the chaotic source text in OP.
Post the program again, embed your source in code TGML tag (press help button on the posting panel for instruction).

 

[mikrom]

It seems that OP posted the source twice.

 

[gummibaer]

Yes, it would be a good idea if you would post your code included in code-tags. It would very much help to read it.
How do you know you are running out of memory? What is the correct errormessage you receive?

Some comments:

As you allways allocate your arrays to the same number of elements (3500) you do not need to allocate / deallocate all the time.

You will receive a memory overflow if any of your obspath-files contains more than 300 filenames. Not because of running out of memory, meaning your computer does not offer enough memory to accomodate your prog's data, but because your filename-array is dimensioned to hold has 300 elements only.

That is all I can see for now - maybe more when your code is easier to read, see above.

Norbert


The optimist believes we live in the best of all possible worlds - the pessimist fears this might be true.

 

[momphy]

@gummibaer
Actually, i check memory use in terminal . after i run this code , memory get less and less . the number of obspath-file is not constant , so i just give a big one to make i can read all file contained .the data i read into syndata and obsdata is more than 3500, but i only need first 3500 . quite confused with memory , why it take more and more memory, only 25MB left .total 12GB.

 

[gummibaer]

Momphy,

indent your statements that are within a loop and post your code between code-tags, so the structure and flow of your prog is better to be seen.

Then get rid of those allocate / deallocate statements. You use the same number of elements all the time, so you do not need them to be allocatable, you may declare syndata and obsdata with fixed dimensions - like in the statement you commented out. At least as a test, if your arrangement of data allocation and deallocation would be the source of your problem.

Norbert




The optimist believes we live in the best of all possible worlds - the pessimist fears this might be true.

 

[momphy]

@gummibaer
at first, i just declare syndata and obsdata with fixed dimension , but i find memory get less and less (top command),then i change it to allocatable and thought this may release memory . the problem is that ,i still have this memory problem .

 

[ArkM]

Try to close opened units (not only reopen them for another external files) then see what happens.

 

[gummibaer]

No, you do not save memory this way. For the short period of time, when your arrays are not allocated, the load on your memory is reduced, but this is of no real consequence to you.

To my experience, a statement like

read (400,*) obsfile

seems to need more memory than

read (400,*) (obsfile(j), j = 1,3329)

Maybe the data are saved temporarily as long as the reading is under way, but I do not know for sure. But it may be worth a try.
Maybe you should replace the full array operations by some looping structure anyway, I do not know how this statement is processed in memory:

corr(i,j)=sum(obsdata*syndata)

Maybe you should replace it by

corr(i,j) = 0.0
do jj = 1, 3500
    corr (i,j) = corr(i,j) + obsdata (jj) * syndata (jj)
enddo

Maybe you would need a little more processing time, but once you know which of your statements is detrimental to your memory consumtion, you may find a better way for to improve your code.

Norbert


The optimist believes we live in the best of all possible worlds - the pessimist fears this might be true.

 

[momphy]

@ArKM
thanks man . such simple , i make a stupid mistake .
@ArKM gummibar
problem solved . thank you guy so much . appreciate both of your time and patients