Parsing a DataBase into Managable Portions for faster queries

[DSJ1967]

Really Long Question...<br>
<br>
I'm currently developing an application that has a rather large database somewhere around 500,000 records divided up into 8 different tables. The problem lies in the fact that the client wants to have an analysis tool where they can have a selection criterion that would potentially look for distinct records (based on an account id) in all 8 tables at one time. Hence &quot;select * from [table 1] where not in(select * from ....&quot;. This would really chug!<br>
<br>
What I came up with may not be the best approach? The database is relativly static (updates monthly) so I decided that I would do some of the queries prior and write the results to a textfile.<br>
<br>
(inside of file looks like)<br>
/1999<br>
[1]<br>
*1<br>
=4 10000001100000002100000003100000004<br>
*2<br>
=1 10000005<br>
[2]<br>
*1 .... etc<br>
<br>
where / & [ & * represent different elements that would be queried against and = represents the result i.e 4 and then 4 account ids. Using Line Input I can then load in only the Account Ids that fit the query but the question is what do I load it into. <br>
<br>
I was looking for something temporary and in memory ie db.createquerydef(&quot;&quot;.. but I am having problems finding something that I don't have to first commit to the database.<br>
<br>
I really do not want to have to create a work table and then delete it after every query. <br>
<br>
Any suggestions?

 

[MikeLacey]

Hi there - what database are you using?<br>
<br>
500,000 rows doesn't sound <b>too</b> bad - not Access is it?<br>
<br>
Anyway - your question.<br>
<br>
My approach would depend upon how many records I was going to read into memory.<br>
<br>
If it's not too many (too many defined here as &quot;it would make it go really slow&quot; you could define a record as a data type and then declare an array of your new datatype. How does that sound?<br>
<br>
If that goes too slowly - have you thought about using the Jet engine to create a local .mdb file to hold your data whilst you fiddle with it?<br>
<br>
Regards<br>
<br>
Mike<br>
<p>Mike Lacey<br><a href=mailto:Mike_Lacey@Cargill.Com>Mike_Lacey@Cargill.Com</a><br><a href= > </a><br>