.FIL files

[Rick C. Hodgin]

Does anyone have file spec information on .FIL files?

I have received a data set from someone using what I'm told is an old dBASE app. It is a large number of .FIL files.

In examining these files internally in a hex editor, I can see they have a defined header structure similar to a DBF file, but without any discernable pattern (at least not to me). I have been able to identify that the two bytes at offset 26 indicate the start of actual data.

The data is not in a fixed format like a DBF file, but is free-flow / contextual. It's intermixed with what look like control codes. It will be like [0xfd03]HELLO[0xfc10]WORLD, etc. Those control codes seem to indicate information about the next field, but I can't make it out.

Any thoughts / info?

--
Rick C. Hodgin

-----------------
PS -- Here's a sample of the information.

Note: This was loaded into memory at the hex offets shown using a custom C program. These are not the actual offset into the file.

The text has been redacted, but each of those locations contained things like dates, names, some IDs or codes, etc. Normal data.

The data at the start of the file is always the hex sequences 10 12 14 16. The rest of the file is exactly as it exists. I have about 180 other files like this. They range in size from a few KB up to several MB. I've been able to go in and parse them to break out the text portions, but the data is not consistent. I think it's describing a report output and those codes 0xfd?? and 0xfc?? etc are indicating things like how may spaces to move, or tabs, or vertical lines, or some such. But I can't figure out the pattern or meaning. Any help is appreciated.

0x720  10 12 14 16 00 03 00 01 00 01 00 01 00 01 0a 00  ................
0x730  00 01 0a 00 00 01 09 c6 00 00 02 01 00 00 00 00  .......Æ........
0x740  00 00 00 06 00 00 ce 00 00 00 00 73 00 00 00 00  ......I....s....
0x750  00 00 00 73 00 00 00 c9 00 00 00 00 00 00 14 6a  ...s...É.......j
0x760  00 00 00 00 00 10 00 01 00 08 00 02 00 00 00 00  ................
0x770  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x780  0a f3 0a f3 03 5a 00 10 00 00 00 00 00 00 00 00  .ó.ó.Z..........
0x790  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x7A0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x7B0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x7C0  00 00 c2 00 02 01 00 0b 00 0b 00 00 00 00 00 00  ..A.............
0x7D0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x7E0  00 00 c8 00 02 02 01 0e 00 08 00 0b 00 06 00 13  ..E.............
0x7F0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x800  00 00 d2 00 02 03 01 10 00 02 00 1c 00 08 00 0b  ..O.............
0x810  00 06 00 13 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x820  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x830  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x840  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x850  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x860  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x870  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x880  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x890  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8A0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8B0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8C0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8D0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8E0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x8F0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x900  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x910  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x920  02 01 e0 00 dd 00 00 00 00 fc 0a .. .. .. .. ..  ..à.Y....ü.Redac
0x930  .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..  tedRedactedRedac
0x940  .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..  tedRedactedRedac
0x950  .. .. .. fc 08 .. .. .. .. .. .. .. .. fc 06 ..  tedü.Redactedü.R
0x960  .. .. .. .. .. .. fc 08 .. fc 07 .. .. .. .. fc  edacteü.3ü.Redaü
0x970  .. .. .. 07 .. .. .. .. .. .. fc 08 .. fc 08 ..  Reü.Redactü.2ü.R
0x980  .. .. .. .. fc 75 .. .. fc 08 .. .. .. .. .. fc  edacüuReü.Redacü
0x990  06 .. .. .. .. .. .. .. fc 06 .. .. .. .. .. ..  .Redacteü.Redact
0x9A0  .. fc 49 .. .. .. .. .. fc 7b .. .. .. .. .. ..  eüIRedacü{Redact
0x9B0  .. fc 13 .. .. .. .. .. .. .. fd 7f fd 7f fd 7f  eü.Redactey.y.y.
0x9C0  fd 7f fd 7f fd 7f fd 7f fd 7f fd 7f fd 7f fd 7f  y.y.y.y.y.y.y.y.
0x9D0  fd 7f fd 7f fd 7f fd 34 .. .. .. .. .. .. fd 18  y.y.y.y4Redacty.
0x9E0  .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..  RedactedRedacte
0x9F0  .. .. .. .. .. .. .. .. .. .. .. .. fd 7f fd 7f  dRedactedRedy.y.
0xA00  .. .. fc 1c fd 32 00 00 00 00 00 00 00 00 00 00  Reü.y2..........

 

[Mike Lewis]

Rick,

I see you have re-posted this question in the Visual FoxPro forum. That makes sense, as that forum is more active than this one, and you will probably get a quicker reply.

In the meantime, let me just pick up one minor point from you post. Re your dump of the header file, you said "View in a fixed point editor". In general, in Tek Tips, the way to present this sort of information in fixed-putch is to wrap it in [ignore]

 /

[/ignore] tags (or use the Code button the toolbar). That will enable the text to be viewed in fixed-pitch, without the viewer having to paste it into a separate text editor.

I just thought I would mention this for the benefit of anyone else who needs to display this kind of information.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Rick C. Hodgin]

Thank you, Mike.

--
Rick C. Hodgin

 

[Rick C. Hodgin]

I was able to figure out the file structure and extract the data. It's comprised of a series of pages, similar to how indexes are stored. Each page is either 512 bytes or 1024 bytes.

    0x900   00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    0x910   00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
==> 0x920  [02] 01 e0 00 dd 00 00 00 00 fc 0a .. .. .. .. ..  ..à.Y....ü.Redac
    0x930   ..  .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..  tedRedactedRedac
    0x940   ..  .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..  tedRedactedRedac

At the 0x920 section there, it shows the value "02" which indicates a data page. A value of "01" indicates an index page.

0x720  10 12 14 16 00 03 00 [01] 00 01 00 01 00 01 0a 00  ................
0x730  00 01 0a 00 00 01 09  c6  00 00 02 01 00 00 00 00  .......Æ........
0x740  00 00 00 06 00 00 ce  00  00 00 00 73 00 00 00 00  ......I....s....

The "01" value highlighted above indicates how large the page frame size is. A "01" indicates a 512 byte page. A "02" indicates a 1024 byte page.

At the start of each page is the one byte code. At the end of each page is a 4-byte structure identifying the data location in the file. The numbers are sequential, but in my observation can be discarded.

Each page is then (512 - 1 - 4), or (1024 - 1 - 4) bytes of data per page. The data pages can be copied sequentially from start to finish to a copy that does not have the control information or the index pages. This provides then a top-down data dump of the raw data in the file, which can then be parsed.

Each row contains fields separated by an ASCII character above 0x80, and a byte following it. I haven't been able to figure out what those bytes following mean as they vary throughout the file. However, immediately after those two-byte sequences are either more two-byte sequences, or real data.

If you extract out the data into a comma-delimited file format you wind up with a full data extract.

0x920  02 [01 e0] 00 dd 00 00 00 00 fc 0a .. .. .. .. ..  ..à.Y....ü.Redac

This portion indicates how long the record is. If you take that length and add 8 to it (for the 8-byte "02 01 e0 00 dd 00 00 00 00" header portion), you arrive at the firs two-byte sequence (0xfc0a here). Immediately after that is the real data, which is shown as "Redac" here.

It proceeds like that for every field in each record.

A very interesting file format. I may have some of the details incorrect, but for the data extracts I was able to do the bulk of the data came through correctly. A few files were invalid and I need to investigate them on a case-by-case basis, but right now I have about 98% of the data extracted.

--
Rick C. Hodgin

 

[Mike Lewis]

A good hack, Rick. No doubt that information will be useful if anyone else needs to extract data from that file type.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Rick C. Hodgin]

We've since come across some other files that we know came from a COBOL system, and they had a very similar file structure.

So we're beginning to think these .FIL files were not actually dBASE files, but were COBOL files with a non-standard file extension.

0x720  10 12 14 16 ...

The 10, 12, and 14 bytes are the same, and the next ones include values up to 0x19. An identical format though with the 512 or 1024 byte page sizes. And the same type of codes throughout. It would be nice to find out what kind of tools would read them.

--
Rick C. Hodgin

 

[Mike Lewis]

Thanks for the update, Rick. I wish I could have thrown more light on this, but, as you have discovered, these files have nothing to do with dBASE. Do let us know if you uncover any more information.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads