File streams, quick intro? 1

[KempCGDR]

Hi, I have a program where part of a record in a file that I need to read in has to be longer than 255 characters. Using records, this is the most I can get up to, so I was wondering if streams let you have longer strings (or some equivalent). The Delphi 3 (I know, I know) help files aren't very helpfull on the subject of streams, so does anyone have any pointers?

The record structure I'm stuck with at the moment is

ItemType = Record
Cat,SubCat,Name:String[100];
Content:String[255];
end;

The content string needs to be about 500 characters in length, might be more when I have it worked out. If I can have dynamic length strings that would be a lot more helpfull.

So, can anyone show me any sample code or anything, thanks in advance.


(remember it's D3, fancy stuff like dynamic arrays aren't in yet, desperately need to get my hands on a newer version)

 

[richardchaven]

D3 does have Huge Strings, so size is no longer a problem.

Once you have a length figured out, the basic protocol for using file streams is:

   with TFileStream.Create(AFileName, fmOpenRead) do
   try
       SetLength(ThisString, SIZE_OF_DATA);
       SetLength(ThisString, Read(PChar(ThisString)^, SIZE_OF_DATA));    // sets the string to the actual length read
   finally
       Free;       //  free the File Stream
   end;

Obviously, you can repeat this at will within the 'try' and load separate string variables.

If this does not answer your questions, please let me know.

Cheers

 

[KempCGDR]

Ok, so if I want to read a string from position 'Position' in the file, I can use

with TFileStream.Create('Filename.ext', fmOpenRead) do
try
Seek(Position, soFromBeginning);
SetLength(StringToReadInto, 500);
SetLength(StringToReadInto, Read(PChar(StringToReadInto)^, 500));
finally
Free;
end;

Some things I don't get:
1) Why do you call SetLength twice? (and what does each call do?)
2) How do I handle the equivalent of record structures with this, ie what if I want to store (for instance) a length 10 string, an integer and a length 500 string together and have multiple copies of that (each one having different data obviously)?

 

[richardchaven]

Instead of

   Seek(Position, soFromBeginning);

just set the Position property:

   Position := ThisPosition;

1: SetLength allocates enough space to read your data and to set the internal property of the string variable (e.g.

Length()

). The first time we use SetLength is to make sure we can read all of the data. The second time is to set the length to the actual number of bytes read (the return value from

Read

.

2: Try something like this:

type
   TDataBlock = record
       FirstString : String;
       OnlyInteger : Integer;
       SecondString : String;
   end;

...

var
   DataArray : array[0..23] of TDataBlock;   // you could use a TList or TCollection as well
   Counter : Integer;

begin
    with TFileStream... do
    try
       for Counter := 0 to High(DataArray) do
       begin
           SetLength(DataArray[Counter].FirstString, 10);
           if Read(PChar(DataArray[Counter].FirstString)^, 10) <> 10 then
                raise Exception.Create('Unexpected EOF reading FirstString of record ' + IntToStr(Counter + 1));

           if Read(DataArray[Counter].OnlyInteger, SizeOf(Integer)) <> SizeOf(Integer) then
                raise Exception.Create('Unexpected EOF reading OnlyInteger of record ' + IntToStr(Counter + 1));

           SetLength(DataArray[Counter].SecondString, 500);
           if Read(PChar(DataArray[Counter].SecondString)^, 500) <> 500 then
                raise Exception.Create('Unexpected EOF reading SecondString of record ' + IntToStr(Counter + 1));
        end;
    finally
       ...
    end;
end;

Cheers

 

[KempCGDR]

Thanks, I'll try all that and get back if I come across any problems, I think I can follow it all now.

 

[KempCGDR]

Ok, so I have

with TFileStream.Create('Data.dat', fmOpenRead) do
try
Position := ItemPos;

SetLength(ItemVar.Name, 100);
if Read(PChar(ItemVar.Name)^, 100) <> 100 then
raise Exception.Create('Unexpected EOF reading Name from record ' + IntToStr(ItemPos));

SetLength(ItemVar.Cat, 100);
if Read(PChar(ItemVar.Cat)^, 100) <> 100 then
raise Exception.Create('Unexpected EOF reading Cat from record ' + IntToStr(ItemPos));

SetLength(ItemVar.SubCat, 100);
if Read(PChar(ItemVar.SubCat)^, 100) <> 100 then
raise Exception.Create('Unexpected EOF reading SubCat from record ' + IntToStr(ItemPos));

if Read(ItemVar.ContentSize, SizeOf(Integer)) <> SizeOf(Integer) then
raise Exception.Create('Unexpected EOF reading ContentLength from record ' + IntToStr(ItemPos));

SetLength(ItemVar.Content, ItemVar.ContentSize);
if Read(PChar(ItemVar.Content)^, ItemVar.ContentSize) <> ItemVar.ContentSize then
raise Exception.Create('Unexpected EOF reading Content from record ' + IntToStr(ItemPos));
finally
Free;
end;

and I'm expecting this to work for my record structure (only needing to read one record at position ItemPos btw), which is now

ItemType2 = Record
Cat,SubCat,Name:String;
ContentSize:Integer;
Content:String;
end;

Before I go through converting everything in my program to use streams (I like them suddenly), is that going to work.

Next question:
How do you write with these, I can see the obvious point of changing

with TFileStream.Create('Data.dat', fmOpenRead) do

to

with TFileStream.Create('Data.dat', [fmOpenWrite|fmOpenReadWrite]) do

but past that I'm completly stuck. I'm going to play around, and if I sort it before any replies appear then I'll post what I got so you can tell me why it's going to crash 9 out of 10 systems

Oh yeah, and last minute thought, if I'm going through the file reading each record, how do I know when I reach the end of the file? I used to use EOF(FileVar) with Assign/Reset/Close.

Thanks for the help so far.

 

[richardchaven]

how do I know when I reach the end of the file?

with TFileStream... do
try
   while Position < Size do   //  ~~ as (not EOF)
   begin
        ...
   end;
finally
end;

How do you write with these?

with TFileStream... do
try
   with ThisItemType2 do   //  dangerous to have nested 'with'. be careful
   begin
      Assert(Length(Name) = 100);
      Assert(Length(Cat) = 100);
      Assert(Length(SubCat) = 100);
      Assert(Length(Content) = ContentSize);

      Write(PChar(Name)^, 100);
      Write(PChar(Cat)^, 100);
      Write(PChar(SubCat)^, 100);
      Write(ContentSize, SizeOf(Integer));
      Write(PChar(Content)^, ContentSize);
   end;
finally
...
end;

If you are re-writing a file and have opened in fmReadWrite, remember to set

Position := 0

before you start or you will just append.

Just to make you life harder <s>, have you considered saving this as XML?

Cheers

 

[KempCGDR]

Not really, because I have no experience using XML (didn't have any using streams either, but I've seen other posts and I knew the general idea). Plus, if I recall correctly, it's harder to use XML stuff in older versions of Delphi (damn my need to upgrade, moved to top of to-do list).

 

[KempCGDR]

Ok, I tried the following to write a record and got an AccessViolation in INEditor.exe (my program).


ItemVar:ItemType;
SizeOfCont:integer;
-----------------------------------------
ItemVar.Cat := edtCat.Text;
ItemVar.SubCat := edtSubCat.Text;
ItemVar.Name := edtName.Text;
ItemVar.Content := memContent.Text;

SizeOfCont := Length(memContent.Text);
with TFileStream.Create('Data.dat', fmOpenWrite) do
try
with ItemVar do
begin
Assert(Length(Name) = 100);
Assert(Length(Cat) = 100);
Assert(Length(SubCat) = 100);
Assert(Length(Content) = SizeOfCont);

Write(PChar(Name)^, 100);
Write(PChar(Cat)^, 100);
Write(PChar(SubCat)^, 100);
Write(SizeOfCont, SizeOf(Integer));
Write(PChar(Content)^, SizeOfCont);
end;
finally
free;
end;
end;

Any thoughts?

 

[KempCGDR]

Should I set position if I just want to append? And also, does the file have to already exist with this code?

 

[KempCGDR]

Ok, worked out most of the probs. Just a quickie, how can I be sure the strings I'm writing to the file haven't got junk after the stuff I put in them in the

ItemVar.Cat := edtCat.Text;
ItemVar.SubCat := edtSubCat.Text;
ItemVar.Name := edtName.Text;
ItemVar.Content := memContent.Text;

lines? I usually have them a set size and simply loop through each position putting a ' ' in each one, but for various reasons, I can't do that here. There's got to be a routine to do it, right? I checked my output file and sure enough, there's junk after the known bits, doesn't help when you try to match it to something.

 

[richardchaven]

TFileStream.Create(AFileName, fmReadWrite) requires the file to exist. Test with FileExists() and do a fmCreate if you need to.

TFileStream.Position will allow you to append (e.g. .Position := .Size).

Part of the Assert(Length(...) = 100) is to insure they are the correct size. You have to turn assertions on in Project | Options | Compiler.

You can always do something like this to fill out the data:

...
          Write(PChar(Name + StringOfChar(' ', 100))^, 100);
...

This does expose some of the problems with fixed-length data files. Delimited data file formats, including XML, get around some of these problems.

Cheers

 

[KempCGDR]

Ok, got it all sorted, thanks for the help guys, I really appreciate it.

 

[KempCGDR]

richardchaven, gave you a couple of stars, just checked the posts and realised that it was just you helping me. Thanks again.