Freeing Memory After Using Large Tables

[boanrb]

Hi friends,

I am using VFP6 on win7. I have a problem with memory leak (?not sure if it is?).

I open a large table then check the vfp memory usage in the Task Manager (processes). I get xKB

I close the large table and check the vfp memory usage in the Task Manager (processes). I still get xKB

Despite the fact that the table is now closed, the memory usage remains the same and unless you quit vfp the status quo remains - and this starts freezing the entire box.

Any clean way of sorting this out? Many thanks in advance.

Benson

 

[Olaf Doschke]

FLUSH
SYS(1104)

VFP caches, what it has read for keeping performance high. If you want to free memory use SYS(1104) and FLUSH. In this case I assume FLUSH would work better, but you may try both.

One more question: What memory usage do you have before opening the large table? Have you looked into that, too? Opening a table alone (eg USE sometable or having it in a forms Data Environment) does not cause memory usage. Only reading in, eg a sql query into cursor.

Bye, Olaf.

 

[boanrb]

Hi Olaf,

Big thanks for your really quick response.

I have 15MB after I start vfp, 15MB after I open the large table, 185MB after I query the large table and 50MB after I close the table.

I have tried FLUSH and sys(1104) but it basically remains at 50MB.

Thanks Olaf, thanks.

 

[GriffMG]

I use this on my systems that need to run constantly:

FUNCTION REDUCEMEMORY
	PRIVATE NPROC,BB
	DECLARE INTEGER SetProcessWorkingSetSize IN kernel32 AS SetProcessWorkingSetSize  ;
		INTEGER hProcess , INTEGER dwMinimumWorkingSetSize , ;
		INTEGER dwMaximumWorkingSetSize
	DECLARE INTEGER GetCurrentProcess IN kernel32 AS GetCurrentProcess
	NPROC = GETCURRENTPROCESS()
	BB = SETPROCESSWORKINGSETSIZE(NPROC,-1,-1)
	CLEAR DLLS SETPROCESSWORKINGSETSIZE
	CLEAR DLLS GETCURRENTPROCESS
	RETURN(.T.)

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

 

[Olaf Doschke]

50 MB is not very much. I could live with that, if it's not constantly getting higher 35MB each time you query the table and flush/purge memory.

You can also control how much memory VFP consumes at max via SYS(3050). That determines max RAM usage, if VFP needs more, it'll swap to hdd, and then things will get awful slow. So you have to balance it to a size that a) is enough for most of the time
b) is available as RAM most probably also at times other usual processes run
c) does not take away too much RAM for other processes

Cases a too low setting hurt are: Rushmore optimization would like to allocate more RAM than you allow, in consequence VFP will not optimize. That is essential when querying large tables, even if you only query a small amount of data from it. INDEX may fail, if not having enough memory to create an index in memory.

You should check more often after sys(1104) and flush, in the end it's Windows, what really does the garbage collection, and that is not done directly all the time. By the way, this is even worse in .NET, if you don't actively call Collect(), on the other side calling it too often causes a performance hit for all client processes. Garbage is normally not done directly, as reorganising the memory stack does take away performance, if done way too often.

So finally, don't be after each byte. If 50MB memory usage remain after purging unneeded RAM, and it won't rise and leak over time more and more. It's a reasonable memory usage, the VFP runtime takes about 10MB, we have GB RAM today.

Bye, Olaf.

 

[boanrb]

Griff,

It worked like a charm! Kindly explain what exactly is happening in the background. Any settings I need to make? Any caveats in the usage of the code?

Appreciated!

Benson

 

[boanrb]

Olaf,

I appreciate your detailed explanation. Thanks alot!

Benson

 

[GriffMG]

Sorry, I have no idea, I have some code that uses an third party add on to process thumbnails
of pdf files and found it had some kind of memory leak. So, I googled the problem and found this
code - 5 years ago - and it worked for me.

Look up the WIndows API SetProcessWorkingSetSize - perhaps you can find an explanation there.

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

 

[boanrb]

Griff,

It's fine chief. Thanks a bunch!

Benson

 

[Mike Lewis]

For what it's worth, Griff's function is basically a way of controlling the number of pages of virtual memory allowed for the app. By passing -1 to the functions (as in [tt]BB = SETPROCESSWORKINGSETSIZE(NPROC,-1,-1)[/tt]), you're telling it to set the number of pages to a minimum (more information here).

As far as I know, there's no VFP equivalent for this function. Personally, I've never used it, because I assume it's better to let the operating system decide how to allocate virtual memory. But clearly it does have its uses.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[boanrb]

Mike,

Thanks for the link and the extra info.

Benson