food for thought - ramdisk

[friend01]

Hi All,

A quick question. I'm pretty darn sure it will wotk but I need to know if any of you have tested this.

VFP app on ramdisk?

Actually, just the tables.

Actaully, here's my question. I have an app that I run 1x a month. It takes 5-6 hours to run. I know it's not that bad since it's only once a month but I was wondering. What if I put (copy) the DBF's (which are huge) on a ramdisk and run the app pointing to the dbf's on the ramdisk? WOuldn't it fly?

I would assume so? Any food for thought? Have any of you tested that?

I think it'ds a pretty good idea. Tell me all your opinions please.

Thanks,
F1

 

[Mike Lewis]

I don't think this is the right approach. Back in DOS days, a RAM drive was a sensible way to use memory above the 1 MB limit. It doesn't make so much sense today. If you use a RAM disk, that's memory that Windows would otherwise use for itself. The most likely result is that more memory would be paged to and from the hard drive, which negates the benefit of the RAM drive.

I think you'd be better off using your energy to figure out why the app takes so long to run. It could be an issue with your index tags, or simply the way the data is organised in your tables.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro tips, advice, training, consultancy
Custom software for your business

 

[friend01]

Hi Mike,

I understand what you're saying and agree to a certain degree. IF I have 16 gig of RAM. and I have 1 table with a size of 2 gig (let's say). I copy the 2 GIG table to the 'ramdisk' which would be created in a size of let's say I do it with a size of 4 gig RAMDISK

Now, from what I undertand, if I do many many "seeks/replaces" on the table on a harddisk, it can only 'seek/replace' as quickly as the speed of the HD (read/write). if I do "seeks/replaces" in RAM, it should technically be lightining speed way so much quicker than a harddisk.

I think the seeks/replaces would so much quicker do to the fact that the access to the table in RAM is quicker that access to a table on harddrive. I'm pretty sure I'm right on that point.

Can anybody test that? or has anybody tested that?

please let me know.

thanks,
f1

 

[bborissov]

2Gig and VFP ))))
You are lucky.
Many SEEK and REPLACE. Could it be done with one SQL command which is Rushmore optimizabe?

Borislav Borissov
VFP9 SP2, SQL Server 2000/2005.

 

[friend01]

Yes, it can be done with SQL rushmore (which is what I'm working on now).


BUT, in the meantime, I was just thinking out loud is all and wondered if anybody has really tried it.

Large table, many fields, many records, many seeks/replces in HDD. Same large table, many fields, many records, many seeks/replces in RAMDRIVE.


I owuld think RAMDRIVE would beat the carp out of HDD.

Who's with me? LOL

let me know if anybody can test that

Thanks,
F1

 

[bborissov]

I forgot about RAMDRIVEs when we moved to Windows.
But you could try to set BUFFERING = 5 for you table and AFTER all SEEK/REPLACE TABLEUPDATE() it.

Borislav Borissov
VFP9 SP2, SQL Server 2000/2005.

 

[friend01]

Yes, I'm aware of BUFFERING = 5 and then AFTER all SEEK/REPLACE TABLEUPDATE(). As I mentionned, I am working on revamping the whole thing and really value all suggestions. Thank YOu.


I was simply wondering if anybody has dabbled the try.


Simply wondering... thanks.


F1.

 

[Mike Lewis]

Well, there's no harm in giving the RAM disk a try, but personally I wouldn't make it a priority.

You mentioned that your DBFs are "huge". Don't forget to factor in the time that it will take to load the files into the RAM disk, and to copy them back to the hard drive afterwards.

Also, remember that, if your program crashes or is interrupted for any reasaon, you will lose all the updates since the start of the run.

I still think you'd be better off spending the time trying to identify the slow bits of the app, and getting rid of them. But by all means give it a try.

Let us know how you get on.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro tips, advice, training, consultancy
Custom software for your business

 

[alvechurchdata]

Before spending your time tuning the app or experimenting with RAM drives it might be worth looking at the age and speed of the disk and network hardware. Given the cost of development time, that might bring more returns more quickly.

Geoff Franklin

http://www.alvechurchdata.co.uk

 

[Imaginecorp]

In theory; your arguement makes sense and it would seem it would be faster during the seeks and replaces ON a Buffered table (set to 5) residing on a RamDisk, but the table will still have to be written to/ retrived from the hard drive...

Why have you not tested this on your own?

 

[Olaf Doschke]

I tried a ram disk, but not for data. Windows is already buffering changes and so as you scan through your tables it should go into memory anyway, no need to have a RAM disk for that effect. It'll only add one more layer, so I'd guess it'll even slow things down a bit. Depends how your processing it. If you mave missing indexes and LOCATE or FILTER, a RAM disk may help, but I'd bet code optimisation would do better.

Bye, Olaf.

 

[friend01]

cool. thanks guys for all your thoughts.

 

[white605]

What about just using one of the new solid state drives?

This old world keeps spinning round - It's a wonder tall trees ain't layin' down

 

[Steve Bowman]

I actualy tried an app and data on a ram drive and on one of the hardware based memory drives i.e. [kind of like a pci thumb drive]. The need was not because it was slow but the process is high transaction and we wanted to sqeeze as much speed as we could. After performing many tests and timings we could not find enough performance improvement to justify doing it so we droped the plan. I was suprised myself, i was expecting more. But it does tell me that read write operations to the disk in windows is pretty efficiant.

Steve Bowman
Independent Technology, Inc.
CA, USA

 

[Olaf Doschke]

Yes, first of all, data needs to be read into RAM anyway. And then it mainly stays there unless you force it to disc with FLUSH FORCE, otherwise there are buffers, VFP does it's caching, OS does caching. And even the HDD does caching.

Bye, Olaf.

 

[kincsem]

Ten years back I wrote programs to analyse thoroughbred horse pedigrees. The pedigree database was 30kb (150,000 horses). The analysis had 90 small FoxPro programs. The first task was extacting a 10 generation pedigree for a horse - 1,022 ancestors in about two seconds. The analysis of these ancestors for one horse took about 40 seconds (re-writes got it down from 3 minutes). I analysed 2,000 horses at a time, and total time was 22 hours.

To avoid constant disk activity I used a commercial ramdisk - one read at the start and one write of the results at the end of each analysis to the C drive.

A re-index of the 30kb file was 1m34s on that old Windows 98 PC, 0m4s on my newest PC (15 tags). I'm tempted to load up those old programs just to see the speed improvement.

 

[Mike Lewis]

That's very interesting, Kincsem.

Was your program written in VFP or in FoxPro 2.x? The reason I ask is that 2.x might not have been able to take advantage of all the RAM in the system, in which case a RAM Disk would be a good move. But I don't know if the same would be true in VFP. If it is, it's a convincing argument.

I wonder if Friend01 is still here, and if he ever followed up his question with some tests. It would be interesting to know the result.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro tips, advice, training, consultancy
Custom software for your business

 

[friend01]

Hi Mike,

Yup, I'm still here. Unfortunately I had no time to follow-up on my question myself. As mentionned, other priorities have hindered me from doing tests with a ramdisk. I like what Kincsem said though. I will definately keep it on the back burner for now and get back to it a little later. When I do, I will definately let y'all know the results.

Thanks guys,
F1

 

[kincsem]

I can't remember the version but I think it predated Visual FoxPro (apologies then for posting it in a Visual FP forum).

The program compared the sire side of the pedigree with the dam side, finding duplicated horses (inbreeding / linebreeding). In places it was going back sixteen generations. I was interested in full and half siblings, position, generation, sex of their offspring and more.

What fired my enthusiasm was picking the winner of the 1998 Oaks at 25/1, and having a (failed) double with it and the horse beaten a head in the Derby the following day (20/1 and 11/1). A few weeks later I had consecutive 16/1 winners (ten hours manual analysis beforehand). That evening I started writing my program.

I'm still interested in pedigrees. Recently I gathered winners of all English, Irish, French group winners from 1900 to date (c. 30,000 results), and am working to fill out the pedigrees. I might extend it to USA, German, Italian. 500+ pedigree books and buying!

I remember the program ran into some looping limit (800,000+) and Microsoft send me a fix to get by it.

The improvements in hard disks, cpu, cheap memory, Visual FoxPro have probably made those workarounds a thing of the past.