slow loading of forms

[Nigel Gomm]

a program suffers very slow loading of forms (do form) when the data is on a network drive. up to 8secs a form.

The same prog works fine as soon as i point it to use data on a local drive (0.3 secs to load the same form).

The program exe is installed on the local drive with all dll,fll and ocxs on the local drive as well.

The temp folder is local. set default and set path are both pointing to local folders only.

The form uses the default data session where all tables are already open. None of the controls on the form are bound to tables (they are bound to data objects). All tables are unbuffered and belong to a DBC.



but here's the funny thing..... set coverage isn't showing any delays; the data is being retrieved very quickly. So a form that takes 6 secs to load may show less than 2 secs of coverage. i.e. the toolbar click (level 2) may show 6secs in the coverage log but the sum of the elapsed time for instructions at the next level down (level3) adds up to only 2secs.

VFP is busy with something behind the scenes.


tried various settings of set refresh. set refresh to 10,-1 at present.

The speed seems to be related to the complexity of the form; but the issue isn't with init or refresh code in the controls. Just for fun i changed the form (edited the scx) to use the VFP parent classes rather than my baseclasses and that made no difference.

I've reduced the complexity at load as much as i can (i.e. tabs in a pageframe don't instantiate their controls till activation) but that isn't an issue when the data is local.

In the past setting PROGCACHE seemed to solve the problem (the .exe is 13.5mb with debug switched off) but recently i've noticed it not having any effect (i wonder if there was a win7 update).

both client PC and the PC with the data are win7 x64 with oplocks and smb2 switched off.

this is being seen on various networks so i think i can discount hardware.

i have
SYS(3050, 1, MIN(512*1024*1024, VAL(SYS(3050, 1, 0))))
SYS(3050, 2, MIN(512*1024*1024, VAL(SYS(3050, 1, 0))))
at startup (and have tried various other settings - thanks Olaf - recently with no effect).

the size of the program makes me think that memory is the issue but messing with progcache and sys(3050) isn't enough. Another smaller program getting its data over the same network from the same server has no such problem (albeit with a smaller dataset). This same program with a much smaller dataset over the network is still slow.

any other ideas?


This is VFP9 SP2 with a couple of the hotfixes applied (09.00.0000.5815).

 

[foxmuldr2]

Is there a database? Are you running stored procedures? Where's foxuser stored? Is the network slow for other operations at the same time (if you run PING continuously in a window do you see longer PING times, etc.)?

Best regards,
Rick C. Hodgin

 

[Nigel Gomm]

yes there's a database. set resource is off. no stored procedures (the delay is not related to data access), the network is fine (again the delay is not related to fetching or writing data - just instantiating a form).

 

[foxmuldr2]

There's a Win32 function called QueryPerformanceCounter(). I would create a little function which calls that value and stores the number along with the code that called it, and then programmatically insert a call to that function in the start of every code-populated object on the form, including referenced classes.

This would give you an exact timing of every event. A postmortem analysis would reveal where your time is being consumed.

Best regards,
Rick C. Hodgin

 

[foxmuldr2]

Something like:

Create a 32-bit DLL with get_time_now() function:

	void get_time_now(s8* buffer32)
	{
		LARGE_INTEGER lpc;

		// Get the current performance counter (tick count,
		// clock cyles / operations executed since the CPU
		// was started)

		QueryPerformanceCounter(&lpc);

		// Use QueryPerformanceFrequency() to find out clock
		// speed, which you can divide the deltas by to get
		// exact timing

		sprintf_s(buffer32, 32, "%I64", &lpc.QuadPart)
	}

And in VFP:

DECLARE get_time_now IN myprog.dll STRING@ buffer32


FUNCTION note_the_time
LPARAMETERS tcCaller
lcTime = space(32)
get_time_now(@lcTime)

* CREATE TABLE timerTable ( cElapsed c(32), cProgram c(128) )

INSERT INTO timerTable (cElapsed, cProgram) VALUES(lcTime, tcCaller)

Best regards,
Rick C. Hodgin

 

[Nigel Gomm]

rick,

more exact than set coverage?

n

 

[foxmuldr2]

Yes. The facilities read clock cycles executed at each call. And if you want to get really fancy, you can start looking at affinity and see what core you're running on, how many threads are running in your process, how much time is allocated to each, and also to record everything in memory without the need of the INSERT INTO code, queuing everything up as a text file which you call a separate DLL function to write the recorded data to a text file you can later import to a table with an APPEND FROM foo.txt SDF command.

Things you do in DLLs allow you to greatly extend what VFP can do. The effects of a real-time running system, with the most limited amount of slowdown, showing the exact sequence of events, recursive calls, etc., visible by exact timings to the start of each thing, this is invaluable information and can be used in ways beyond those available to VFP by itself.

If you really want to get fancy, create a separate running process and have a small DLL which looks for that process, and then creates a Named Pipe between it, sending commands to the external process, which allows you to examine (in real-time) the running nature of your VFP program.

The possibilities are limitless.

Best regards,
Rick C. Hodgin

 

[Olaf Doschke]

coverage log is only covering lines of code executing. As you say the form uses the default data session and loads no further tables and doesn't bind to tables, coverage isn't covering all activity. You already know, as you see a diff betwenn coverage summed time and the load time you experience.

Just one thing to check: Ar3e you starting coverage log before opening the form, or in it's init? In a form Init is not the first thing, event order starts with Load and there is activitiy even before load, of you set form properties to =expression, those are evaluated while loading the form and it's load event before executing that load event code.

Bye, Olaf.

 

[Nigel Gomm]

Olaf,

coverage is started before the toolbar command button click...so yes, before the load event.

n

 

[craigber]

After reading the entire thread, I still think this is data related.

Are you loading the tables directly or using views or queries?
What is the code that access the data. Look in the data session code, load and init.
Are you doing a SELECT <alias> or USE AGAIN?

There are things to think about. For example
USE Table IN 0 ORDER TheTag
is slower than
USE Table IN 0
SELECT Table
SET ORDER TO TheTag

Craig Berntson
MCSD, Visual C# MVP,

http://www.craigberntson.com/blog

 

[Nigel Gomm]

Craig,

no data session code. All tables are manually opened at startup in the default session and stay open.

=ADBOBJECTS(laTables, "TABLE")
FOR EACH lcTable IN laTables
IF USED(lcTable)
USE IN (lcTable)
ENDIF
USE (lcTable) IN 0 SHARE
NEXT

the code to access the data resides in a custom data object. It has a .fetch() method that seeks on an indexed PK. I can see that (and all explicit data access) in the coverage taking a minimal amount of time (more than if local but still acceptable). That's not to say going through this process hasn't given me the opportunity to further optimize.....


I keep comparing it to the first time i saw this symptom (or something very very similar) a couple of years ago. Simply putting PROGCACHE=250 into the config.fpw fixed it instantly (a 10 sec form load became a 1 sec form load with no other change). But that may be confusing me more than it should. I'll keep picking away at the, err, further optimizations then see what i'm left with.

After reading the entire thread

yes, sorry about that ....

thanks

n

 

[tamayok]

This is an old thread but I stumbled upon this potential solution while searching for clues. Apparently, there is very significant improvement to the speed of networked VFP applications by disabling the network card driver Interrupt Moderation setting.

[URL unfurl="true"]http://stackoverflow.com/questions/6961516/foxpro-vfp-create-sql-view-slowness-on-windows-7[/url]

 

[Olaf Doschke]

OK, tamayok.

But I don't see how the stackoverflow discussion is about Interrupt Moderation. The answer markes was about write cache of the servers raid not being in effect.

And nigels solution was to higher the PROGCACHE, which is not a data cache, but program cache. That is memory foxpro allocates for programs and memory variables. I'd suggest trying PROGCACHE=0 for dynamic memory allocation, as does the VFP OLEDB Provider.

In regard of network what I have seen slowing down performance is negotiation of bandwidth, NICs should be configured to the max speed the network allows and not negotiate the speed, as foxpro does lots of smaller packets.

Also looking back at what Nigel said, he said: the data retreival is not the problem, but he also said tables are opened by the de. While that just opens tables, it does validate them, and that causes quote some back and forth between client and server.

Last and least important:

Version number 09.00.0000.5815 is VFP9SP2 without any hotfixes applied. Hotfixes change the last number to 6303, 6602 and 7423 respectively. You just need the last hotfix, as it's cumulative, no matter what MS writes about hotfixes not being cumulative in general. There only is one vfp9r.dll and it can't be patched to only have the 1st, 2nd, 3rd hotfix or any combination, there would need to be 8 dll versions to make that possible. The runtime dlls and IDE exe were changed in chronological order, cumulating changes.

Bye, Olaf.

 

[Nigel Gomm]

Thanks for the interest... the problem still exists.


The situation is a bit better thanks to other improvements but still that difference between local and networked data that can't be attributed to explicitly fetching the data. And another, much smaller, program with a similar architecture on the same network shows practically no difference between local and networked data. Hence my belief it's a memory issue.

I was aware of the Interrupt Moderation setting and that hasn't made a difference here (or on other networks).

I tried PROGCACHE=0 (as opposed to just removing the line) but that added approx .5 sec to the form load (as compared to PROGCACHE=260).

Just one point to emphasise.... there is no DE on any of the forms. All tables are opened in the default data session at startup and stay open throughout. All forms use the default data session.




n

 

[Nigel Gomm]

tamayok... thank you for bringing this up again and making me revisit.


The problem seems to be grids. (Most of the form's tabs instantiate their controls in the activate code but a few grids were left around ..... )

Even though each form's load event does a select 0 and the grid baseclass's init sets the recordsource to a local empty cursor to prevent them picking up the current table as their recordsource.

Changed the baseclass's recordsource (at design time) to "griddummy" and then, at startup,

create cursor griddummy (dummy c(10))


and that reduces one form's load time from 5 secs to < 1 sec. (That form had 3 grids at design time).


Don't know why i didn't think of that before.


n

 

[Olaf Doschke]

Nigel,

OK, grids load data, but only as much as they display. If that is causing so much delay how many of them do you have and what do they load? Loading whole records with Gen or Memo can also mean loading a lot of unneeded bytes.

Besides the solution to bind to a dummy empty cursor, it's much better to make use of form.bindcontrols. Set this .F. at design time, and at runtime set this .T. at the end of init, when you have all real grid aliases opened. You then don't need to switch from a dummy cursor to the real data, which in itself can couse trouble in columns losing their code by being reconstructed, etc.

I never ever again used the approach of empty dummy cursors, since there is form.bindcontrols. If that is .F. all controls don't care for their row/record/controlsource but keep their design time configuration. That's very important especially for grids, enabling you to delay the load of their data to form init, when you can apply form init parameters, etc.

You can even delay the form.bincontrols=.t. to the first activate or even have it in a click of a filter button or anywhere else you want to show data first. It doesn't hurt, if you do it later. Just one thing: It only works once in a form life, you can't unbind controls by resetting that to .f., but it should be enough to get through initialisation problems.

Bye, Olaf.

 

[Nigel Gomm]

Olaf,

i had tested with .bindcontrols a while ago and it didn't improve things.

Some screens have 3 or 4 grids on tabs in a pageframe. None of them have design time formatting - typically when the tab is activated i fetch the data into a local cursor, assign the recordsource and set the headers and columncount.

In retrospect what i think was happening is that when .bindcontrols was set true the grids on tabs latched onto the current alias (because tab not yet activated and the grid is not yet bound to the local cursor) and the overall elapsed time appears much the same.

I may test some more but for now the forms are much (MUCH) snappier and the impact to my code was minimal.

n

 

[Olaf Doschke]

If you use the bindcontrols approach, of course grid recordsource MUST be set first, not afterwards. That's the whole point about it, you prepare both grid binding properties and columns etc. to what you want to bind, open the data in the sepcified alias, and then say "go, bind yourself!".

What you're doing, when you set recordsource at runtime is causing a new grid inner object structure to be created with all defaults, eg fields in alias order, number of columns matching the alias number of fields. If you then set columncount down, you perhaps let VFP create a lot of objects (columns with header and text1) in the first place to then destroy them. I wouldn't think this is causing a time dealy, though, as you can programmatically generate quite a bunch of controls without taking much time.

Bye, Olaf.

 

[Olaf Doschke]

>what i think was happening is ...the grids on tabs latched onto the current alias (because tab not yet activated and the grid is not yet bound to the local cursor

Yes, that's what happens. You can't have an unbound grid, it will always grab what it finds, no matter if it's in a tab not yet visible or not. Then you're perhaps better off creating the grid itself just in time, don't even start with a blank grid, start with a blank pageframe page. Then there is nothing grabbing data.

Bye, Olaf.

 

[Nigel Gomm]

i know what you're saying but i prefer (or at least did) to have the grid on the tab at design time so it can have specific event handlers (typically dblclick, rightclick and resize) but not fetch the data till (and if) they click on the tab.

I know bindevents can do this or add the grid to a container at design time and addobject() on click... but this is minimum impact to existing code so i'm happy.

n