Recover data from .dat and .idx 1

[DeMoNiC810]

Hi! At first, sorry for my english. My problem is that I have any files from MicroFocus Cobol (.dat and .idx) and I need to export data from Cobol data files to txt or similar. I need to do this using C/C++. I don't have the logic structure of that files...

I will be very glad for any help you can offer me to solve this problem.

Here is a sample of files:

http://www.megaupload.com/?d=6UAWXJKI

 

[mikrom]

Hi DeMoNiC810,

Similar problem (but for RM/COBOL) was discussed in thread

http://www.tek-tips.com/viewthread.cfm?qid=1508458&page=1

and the approach how to solve it was showed here

http://www.tek-tips.com/viewthread.cfm?qid=1510638&page=1

 

[webrabbit]

R/M COBOL indexed files are not at all like Micro Focus COBOL indexed files, so the above referances are relevant only for decoding the data within the records.

The .IDX file contains indices only, no data, and can be ignored.

The .DAT file is composed of variable-length records; each record is composed of a 2 or 4-byte record header, followed by the data, followed by 0-3 slack bytes.

The first 4 bits of the record header are indicator bits, if the bits are 0100, it is a good data record; otherwise, it is a deleted or non-data record. The following 12 or 28 bits are the record length. If the maximum record length exceeds 4092, the length of the length field is 28 bits, otherwise it is 12 bits.

The total actual record length is always a multiple of 4 bytes; the record length field does not account for the slack bytes at the end of the record, but it does account for the record header.

The first record is the file header record, it has indicator field of 0011 and a record length field of 128. It contains file attribute information, including the maximum record length.

 

[mikrom]

I looked at the data files in HEX-editor. Webrabbit is right: the MF COBOL files are different from the RM COBOL files.
IMHO, the simplest way would be to write an simple MF COBOL program which print all the data into CSV or similar TXT format.
If you have licence problems with the MF COBOL compiler, then you can try the free personal edition of NetExpress, which is available here

http://www.microfocus.com/Resources/Communities/Academic/shop/index.asp

 

[webrabbit]

I have a MF COBOL program that generically reads a .DAT file. It uses MF byte-stream (binary) file processing logic, and could be easily re-written in C or assembler, or any language that can read binary files.

 

[mikrom]

Hi webrabbit,
It's very interesting !
Can the program be compiled with the free NetExpress Edition? (i.e. it should have less than 2200 lines)
If it would be possible, you could post it here

 

[webrabbit]

Sorry, Tek-Tips error trying to post it. Send me an e-mail at jamesfairfield at cox.net, and I will send the program to you as an attchment

 

[mikrom]

Hi webrabbit,
Thx for the prog - a star to you.
I will try to compile it with the free NetExpress COBOL.

 

[DeMoNiC810]

Thanks for your answers. webrabbit I would appreciate it if you would send me the program
I will send you an email.

 

[mikrom]

Hi webrabbit,

I think I compiled your program succesfully with NetExpress, but I don't know, how to run it.
Does it need an command line argument or is it a dialog program?

 

[webrabbit]

As I said in the e-mail, it is a sub-program. You have to write a driver program that does the main processing.

When you call the sub-program, you pass it two parameters, the first being the group name in IO-BLOCK.CPY with various fields within it initialized, the second being the I/O area.

Before calling the sub-program, you must first set up the various fields, one being always the I/O request type; OPEN, CLOSE, READ, etc.

For OPEN, the I/O area must contain the path name of the file to be read. For READ-NEXT, the I/O area will receive the next record. You must set the length of the I/O area in the first parameter before each call. The I/O status field will contain the status of the I/O request after the call.

If the I/O status is other than 00 (Normal) or 10 (File at end), the sub-program will attempt to call another sub-program to display the error and request user action. As you don't have this other sub-program, the MF COBOL system will instead display that fact as an error.

Note that after each READ, the sub-program will change the value in RECORD-LENGTH from that set by the calling program to the number of bytes just read.

 

[DeMoNiC810]

Then, can I read my files with this sub-program? Sorry, but my knowledge about cobol is very poor...
Thanks webrabbit

 

[webrabbit]

Basically, you CALL this sub-program instead of the standard COBOL statements such as OPEN, READ & CLOSE.

 

[DeMoNiC810]

Sorry but I don't know how to call your program in a main process... Can you help me?

 

[webrabbit]

I have rewritten the program as a stand-alone main program with no copybooks.
I have included notes as to where to put your record processing logic.
If anyone wants a copy, send a request to the above email address.

 

[Truusvlugindewind]

Whenever you must access MF files from another language than MFCobol, you can call the MF I/O routines separately.
This module is called extfh (

http://docs.hp.com/en/64/fh/fhexfh.htm)

 

[webrabbit]

MF I/O routines only work if you know the file attributes.

 

[Truusvlugindewind]

Hmmm... The topic-starter mentiones .IDX files, so the organization is indexed. This leaves record-length and record layout.
Just trail&error until you can retrieve a whole record and then analyse the hexadecial structure:
PIC X fields should be readable in normal format
PIX 9 packed-deciaml should be readable in hex format
PIC 9 binary should be "decoded" using the windows calculator (view scientific and switch between the 'hex' and 'dec' radio-button.

[memory-lane]
Just like the old mainframe dumps or ditto-output
[/memory-lane]

 

[webrabbit]

Truusvlugindewind, record length is only one of the file attriutes that are significant to th MF I/O routines. Ignoring the indices, as you can by defining the file as a variable length RECORD SEQUENTIAL file, there are the maximum and minimum record lengths, and one or more other attributes which I have never been able to determine.

My routine bypasses all that and assumes a maximum record length of 4096. All the other points you mention had already been discussed, if you had actually read the posts.