Load StringGrid from Stream

[atcbill]

Hi. I'm trying to load a TstringGrid from stream and having difficulty. I originally loaded the stream to a tstringlist then to the grid which worked fine but was REALLY slow. Any ideas on how to do this or speed up the process. The data I'm loading is a text file that has about 1200 lines in it. A sample of the lines are:

00:12 SI 1F TQ RJ 00:12 1F
00:15 SO 2D LS 00:15 2D
03:45 CO 2U 2Z 03:45 2U

The text is not space delimited but by column numbers, ie, data1 is colums 1-5, data2 is columns 7-8, data3 is colums 10-11, .....

Each text file is for the day of the week so when I load up a months amount, it takes almost 30 minutes on a fast computer via the tstringlist method. Any thoughts or help would be appreciated.

atcbill

 

[towerbase]

Do you really need a TStringGrid? TDrawGrid is usually much faster.

Do you set the number of rows in your StringGrid once (and only once) before you start reading the data into your StringGrid? Changing the row count can be expensive in time when the grid gets large.

I am not sure why you mention stream. Why not do a StringList.LoadFromFile ?


Andrew

 

[svanels]

My approach:
1 )Read the a line in a string variable called LineString
Readln(Linestring, MyTextFile);
2) Take out the data with copy function
data1:= Copy(LineString,1,5);
data2:= Copy(Linestring,7,2);
etc....
3) Store the data in the StringGrid

4) Wrap it up in a loop with eof



Steven van Els
SAvanEls@cq-link.sr

 

[atcbill]

Andrew and Steven, thanks for replying.

Andrew, I need a stream because I am loading multiple text files to a list. What I'm doing now is reading a file into a tfilestream. Then I use copytostream to copy it to a tmemorystream. That way I can load several text files in a row and get them appended one after another into the memorystream. After they get to a memorysream, I load them to a tstringlist with loadfromstream command.

The reason I'm loading to a grid(and I should have mentioned this in my original post), is that I'm trying to find a faster way to load the text data into an Access database. When I load to the database directly from a stringlist a line at a time, it takes forever. What I was trying to do above was to load the stream into a stringgrid and see if it populated the database quicker.

Steven, I tried what you advised initially by just opening the file, reading a line, then copying it into my database. It was unbelievably slow. The data I posted above is what I load in. Each line is only 29 characters long. It takes about 20-25 seconds to load the database with a 1100 line text file! When I put run the data for a month ( 30 files times 1100 lines) it takes over 15 minutes. There has got to be a better way do this. I'm still working on it and thanks again for the responses.

Bill

 

[svanels]

Increasing the input buffer to about 1024 K, helped in 1989 on DOS systems.
At that time I was mounting textfiles to export data to a wordprocessor. I reduced the generation of 60 page report from 3 minutes to about 30 seconds. On a 286 with only 2 floppies.
20-25 seconds to load 1100 records in a database doesn't look bad on first sight.

I would look at:
1) time to process the textfile, can be increased but I suspect that this is the fastest part. Maybe it will be reduced from 8 to 2-3 seconds.
2) time to enter the data in the database, probably this is time consumer, because it involves the windows operating system
3) The transition from 1 textfile to another, probably insignificant, but needs examining
4) Other bells and whistles: probably the villains
a)Are you updating some visual controls (grids, dbedits) etc. together with the database?
b)Consider the transition textfile --> database --> visual update, no visual component interface, except for speed measurement in the first part
c) Create a timer procedure (No TTimer component !!!) to measure speed, someting like:
clockon; myprocedure; clockoff;
d) Use the procedure ProcessMessages to eleminate windows messages interference in critical parts.

Good luck Steven van Els
SAvanEls@cq-link.sr

 

[towerbase]

It's not clear to me why you need to concatenate all the text files before writing them to the Access database. Why not load one file at a time into a StringList and then insert the records from the StringList into the database?

Maybe I don't understand the problem.

Incidentally, are you using ODBC to get at Access? I have found this to be very slow. The native Delphi driver for Access is much quicker.

Andrew