Fastest way to read large txt files 2

[AgentM]

Hi

My asp code has to read a .txt file which is approx 3 MB in size.

The data on each line has to be filtered into a database.
I am reading the file line by line and using the mid function to separate the data.

This takes such a long time that IE stops responding.

Does anyone have any ideas , abt how to make the process faster?

Any ideas appreciated.

 

[foxbox]

to make it faster i would import the txt into a database file format, preferable with the correct structure. in my old dBase days that was very easy:
APPEND FROM <txtfile> SDF.
(this will only work with fixed fieldlengths; i presume the txt it is not delimited.)



br
Gerard

 

[vasah20]

see this link

http://www.4guysfromrolla.com/webtech/010401-1.shtml

that should speed things up for you

leo

 

[AgentM]

Hi
Figured it out.....
Used ActiveX DLL and macros (though not a good soln)in Access

Works great.

Thank you'll for all the help

 

[Swany]

Vasah's link is a good one.

You could also open the file using an ADO stream object. Most people don't know that ADO streams handle text files very well.

Also, are you sure you need to use MID to split the data? It is a standard delimited file (field1[delim]field2[delim]...)?

If it is, then use the split command to break it up into
an array..

const adReadLine = -2
const adOpenStringAsync = 1
const adTypeText = 2

filename = &quot;Myfile.txt&quot;
'create a stream object
set file = server.createobject(&quot;ADODB.stream&quot
'Open a file
file.open filename, adTypeText, adOpenStringAsync

do

in_line = file.readtext(adReadLine)
if isNull(in_line) then exit do
'Assume record is pipe (|) delim
record = split(record, &quot;|&quot

'assume dest database and recordset was already opened
rs.addnew
for i = 0 to ubound(record)
rs(i) = record(i)
next
rs.update
loop

file.close
set file=nothing