** File size never larger than 4GB ** 1

[fabiousa]

Guys,

I have a system running on production and it's a simple COBOL program which is supposed to read an Input file and, after doing some calculations, generate an Output file.

This program works with smaller input files, which generate small output files correctly. I just multiply the record length by the number of records and get the total number of bytes.

The thing is, whenever my Input file cointains a large number of records, the size of the Output file is never larger than 4,294,969,056.
This happened for example whan I had 3 million records and 4 million records.

I can assure you the code is ok because it's been used for years with smaller files.

Is there any limitations on Windows NT?

thanks a lot,

Fabio

 

[Dimandja]

Hi Fabio,

I think your code is fine. But, on a 32-bit machine, addressing is limited to 4GB.

High-end servers and the 64-bit version of Windows XP support a lot more of system virtual address space (at least 128GB).

There are software solutions (used in some Windows 2000) that were developed after Windows NT went out.

Dimandja

 

[fabiousa]

Thanks for the explanation but I was told that if I re-compile my programs using an option called FILEMAXSIZE=8 then I will be able to address up to 8Gb.

Does anyone know how to set this option?
I have tried to find it on my Net Express but I am a simply user.

Thanks a lot,

Fabio

 

[Dimandja]

I have not used this option, but you could check these links:

http://www-ufrima.imag.fr/PLACARD/ECOLE2/ASG/cobol-CD/html/books/fsadvo.htm

http://www.microfocus.com/academic/pcobhelp/gen-01.asp

 

[MKuiper]

Fabiousa,

it is not a compiler option, you do not have to recompile anything. It is an extfh (external file handler) option. Check your system for a file called "extfh.cfg" somewhere, and add the line "FILEMAXSIZE=8" under the [XFH-DEFAULT] section if you want all your files affected by this option. If you only have one file which is so big, you can also add a section for just this file. if the file extfh.cfg is missing, which is possible, create a file extfh.cfg somewhere, which contains the following two lines for all files:
[XFH-DEFAULT]
FILEMAXSIZE=8

or for just one file:
[XFH-DEFAULT]
[C:\THIS\IS\THE\BIGGEST]
FILEMAXSIZE=8

Next you tell the runtime where the file is, by setting an environment variable EXTFH:
SET EXTFH="C:\SOMEWHERE\EXTFH.CFG"

Note: This setting does not change existing files. If you want to convert them to the big format (which is called IDXFORMAT"8" format), you must do so by using the rebuild utility from the commandline or "Data Tools" from the IDE.

Next a few warnings out of the documentation:

Warning:

Do not set FILEMAXSIZE to 8 under any of the following circumstances:
You are running applications on either Windows 95 or Windows 98. (Under this circumstance, if you do set FILEMAXSIZE to 8, the File Handler ignores this setting and behaves as though FILEXMAXSIZE was set to 4.)
You are running applications that access FAT file systems, since such file systems do not support file addressibility greater than 32 bit file addressing. (Under this circumstance, if you do set FILEMAXSIZE to 8, the File Handler ignores this setting and behaves as though FILEXMAXSIZE was set to 4.)

You are sharing files between applications running on Windows NT and applications running on either Windows 95 or Windows 98
You are sharing files with applications using earlier versions of the Micro Focus Cobol product that only supported 32 bit file access

Marcel

 

[fabiousa]

Thanks a lot for all your answers.
I am not sure if I will be able to test the code on my environment.
What I did was, I went to:

PROJECT PROPERTIES - ENVIRONMENT (APPLICATION)

Then I filled the following fields:

VARIABLE= EXTFH
VALUE = "C:\SOMEWHERE\EXTFH.CFG"

Clicked on SET and then OK. Is there anything else I should do?

Thanks a lot again!!

Fabio

 

[MKuiper]

If the file is created by your application, this must be it, provided the location C:\somewhere\extfh.cfg is correct (which seems quite unbelievable to me), and the contents of extfh.cfg is also correct.
But if the file already exists, it must be converted by the rebuild utility or by the IDE in the "Data Tools" menu. Once you have the file in IDXFORMAT"8", you don't need the extfh.cfg anymore, the filehandler will recognize the file format and adjusts itself to that when opening the file. Marcel

 

[fabiousa]

Thank you guys for being so patient with me.
The thing is that I come from a complete different plataform and language (OS/390 - assembler) and I think this Net Express thing is to complicate.

If this is not a compiler option, when I move my application to another PC, then I need to change something else somewhere, right?

I created a file as you said (EXTFH.CFG) with the content:
[XFH-DEFAULT]
FILEMAXSIZE=8

I placed it on the following directory:
D:\Program Files\MERANT\Net Express\Base\LIB

Then I SET the variable EXTFH.
If I take a look at project properties APPLICATION (environment variables), that's what I have:

VARIABLE= EXTFH
VALUE= D:\Program Files\MERANT\NetExpress\Base\LIB\extfh.cgf

I recompiled all the programs on the project and rebuilt it.
No success. When my OUTPUT file reaches 4.194.304 bytes, it stops.

Did I do anything wrong?
I just want to write a file larger than 4GB.

 

[MKuiper]

Fabiousa,

When you move your application to another PC, you have to install Net Express Aplication Server (the runtime) on the other pc, copy your program files to it, create a extfh.cfg file and set the environment variable EXTFH to point to that file.
Are you talking about an indexed file? I think the FILEMAXSIZE parameter is meant for indexed files only (not sure about this).
Does your pc run NT, 2000 or XP? Do you have FAT filesystem?

If you write:
VARIABLE= EXTFH
VALUE= D:\Program Files\MERANT\NetExpress\Base\LIB\extfh.cgf
the VALUE part contains a typing error, extfh.CGF instead of .cfg. Is this error in this post only, or did you copy/paste it into your post (The file will not be found in that case, and the settings will not be applied)?
You write about a filesize of 4.194.304 bytes. That's 4 MB and not 4GB! If it is 4MB, there must be something else what is wrong. Don't you accidentaly try to write duplicate key records, which prevents the file from growing any bigger?

If this is all correct, i don't know and hope somebody else in this forum can help you. Otherwise you must consult Micro Focus support. Marcel

 

[fabiousa]

Marcel thanks a lot for your explanation.

The cgf was a typo and I meant to say 4.193.304KB.
I think I followed your instructions correctly.

My file is not indexed.
Actually, this is a very old application which I need to change just to be compliant with the new volumes.

I am wating for some Micro Focus support (don't know how fast they are in Brazil).

Anyway, thanks again.

 

[MKuiper]

Fabiousa,

Just for my curiousity: would you be so kind to let me know in this forum what the answer from Micro Focus support is?

Thank you
Marcel

 

[fabiousa]

Marcel

I have not tested their solution yet.
What they told me to do is:

1) Set the EXTFH variable path
2) The extfh.cfg should be located on the same directory where my output file will be written
3) One line before my SELECT statemant I should include:
$ SET IDXFORMAT(8).....

hope it works....


later,

Fabio

 

[saginaw]

We are in the process of testing an application with smaller files and were planning to do the final test with the larger production files in the very near future. We were watching this thread and decided to run some initial testing with larger files. At first, we ran into the 4-gig limit and after a little research, frustration and trial-and-error we have it working on our Windows/2000 platform on a drive that is formatted for NTFS.

Here is what we had to do...

Our EXTFH.CFG file contains the following two statements.

[XFH-DEFAULT]
FILEMAXSIZE=8

The EXTFH.CFG file was located in the same directory as the file being created.

Our Net Express is Version 3.1 with Service Pack 1 (SP1). From within the Net Express project we selected "Project" and "Project Properties" and added the FILETYPE(8) directive to the Project Directives. The FILETYPE(8) is apparently a replacement for the IDXFORMAT(8) directive according to the Net Express documentation. Also, it appears that setting this directive in the project can be used in place of the $set statement in the COBOL program. It appears this "large file" support is both a compile and runtime dependent issue.

It took fifteen minutes to create an 8-gigabyte, keyed-indexed file of 30-million records.

We also encountered another frustration when we put the display statement in the COBOL program to help in debugging. The display items did not display when we were running the .EXE (Note: we did the EXE to test performance.

To correct this and have the display item display to the screen we set the link options to static, single-thread, character.

In Summary, it does work when all the pieces are properly coordinated. And YES... it is a bit confusing and frustrating. If you are interestined we are "scrubbing" the sample code and documenting this effort for future reference. We will make available all the documentation, source code and configuration settings upon request.

Saginaw
helpdesk@simotime.com

http://www.simotime.com

 

[MKuiper]

Thank you, Fabiousa and Saginaw for the information. Marcel