Fortran Program Dealing with Very Large Arrays

[ajrust]

I am writing a program in fortran which reads information from some raw binary files, and does some arithmetics with the information.The meat of the program works fine, but there is an Issue.The Original file was around 13000x21000 pixels, which when i created and run, the exe was too large so it did not work.I solved this problem by using Allocatable arrays:

REAL, ALLOCATABLE :: array1,

ALLOCATE(array1(13081x21272))

to allocate the space.

This worked fine,but not I need to include another file which is even larger (19200x22800). I defined an array in the same way

REAL, ALLOCATABLE :: array2,

ALLOCATE(array2(19200,22800))

but when I run the program, there is a run time error which says there is not enough virtual memory available.

Note: I am working on a 32bit machine.

If anyone could help me solve this program it would be greatly appreciated.

thanks very much,

Andrew

 

[SmeeBobFred]

I don't know much about memory allocation in FORTRAN but I do know that a 32-bit program cannot usually access more than 2 GiB of RAM at once. Although a 19200x22800 array wouldn't reach that 2 GiB limit (1670 MiB), if you're using it at the same time as the 13081x21272 array, then together they'd be over this limit (2731 MiB) and you'd get a memory error like that.

I think you'll have to either not have both arrays allocated at the same time or compile a 64-bit version of your program and run it on a 64-bit machine.

Another option is to enable the "/3GB" switch on your operating system (assuming it's Windows) but I don't think all programs can take advantage of this so it may not work with the compiler you're using. If you want to try this, see here for instructions:

http://usa.autodesk.com/adsk/servlet/ps/dl/item?siteID=123112&id=9583842&linkID=9240697

 

[salgerman]

I don't know much about this either, or about pixels...what does one of those pixels in your binary file look like, you know, if it wasn't in binary? is it just one real number? or something else? how much memory do you need to store one of those pixels?

What I am trying to find out is if the maximum value that a pixel can take is not very large, then you do not need to declare your array as default REAL, maybe you can reduce its size to REAL*4 or if the data is integer, you can further reduce how much memory you need that way, etc.