'Exclusive' Network Bug in VFP 6? 2

[alan232]

Hi All;

I hope somebody can help me with this. I have a stay-resident program under windows xp, that reads/writes to VFP 6 tables. No problem. Here is the problem, if I first open those tables with vfp 6 in shared mode, then have the stay-resident progrm (mEnable) access those files, then close them in vfp 6, I can't get vfp 6 to reopen them exclusively. However, if I completely exit vfp 6 back to the desktop and then restart vfp 6, I can once again place an exclusive lock on the files.

Since the stay-resident program is running all the time and its only a matter of exiting and then reentering vfp, I am at the conclusion that there is a bug in vfp 6.

To be more exact, this scenario fails 100% of the time--and it shouldn't:
1. open files with vfp in shared moded.
2. let stay-resident program read/write to files.
3. close files with vfp 6
4. try to open files again with vfp 6 in exclusive mode--won't happen!

Any ideas anybody?
Any help is appreciated...having to completely exit then restart vfp is not a good option.
Alan

p.s. I'm being told to drop vfp in favor of sql, sybase, or oracle--I really don't want to do that.

 

[alan232]

Hi Mike;

I wish it was and I could. However, I don't have access to menable's code--its a visual ce product. However, according to their tek, it works well with sql and sybase. My general feeling is they are probably correct and I'm either doing something wrong, or missing a setting, or vfp 6 is in error.

After reviewing some of the older forum messages, it appears others have had similar problems with vfp and it was suggested that I issue a 'close all..clear' sequence, but in my app I can't do that ...here is why...

The TSR is the 'server' part of a wireless connection between my PDA and the desktop. The desktop app I wrote needs to be able to read/update/write to the same files being kept in sync with the pda. The app I wrote is triggered by an internal timer and also reads data from other computers, bringing everything together for my pda (which I use throughout the office). Long story short, I can't change the TSR, and I can't close the files...

mEnable read/writes to VFP via a ODBC driver--I can't seem to change the settings on this either.

I have to wonder if upgrading to vfp 9 would solve the problem...?

I'm sure open to suggestions.

My goal would be to prevent 'partial readings' by the PDA by using 'exclusive' and to prevent 'partial readings' by vfp by again using 'exclusive' on the files. But once I issue: 'Use myfile Exclusive' and it fails, further attempts also fail--even if 'Use myfile Shared' is used before trying again. and fLock() does not seem strong enough to prevent file reads by the PDA..

I'm also wondering if, instead of the free tables I'm currently using, if I put them in their own database and switched to transactions if I could not then get it to work?

Any ideas?
Thanks for any help
Alan

 

[DSummZZZ]

I don't think you have to worry about a bug being in VFP 6. VFP has been quite stable in it's record and file locking since - well, forever. The only real issues I've ever had with locks came from using Novell.

Anyway, it seems more likely that menable via ODBC is keeping the tables open. You won't be able to lock the header/use them exclusive as long as this is the case. You should be able to lock records still though.

However, according to their tek, it works well with sql and sybase.

Those definitely won't be using tables exclusively either.
It sounds like you're going to have to rethink the strategy.

Can you elaborate a little more as to how the records are processed? We may be able to come up with some ideas.

-Dave Summers-

Even more Fox stuff at:

http://www.davesummers.net/foxprolinks.htm

 

[alan232]

Hi Mike and Dave:

I sure appreciate your help! This is a very tricky problem for which I have tried everything I can think of. I'm game to rethink anything!

Let me digress a moment and give you the background setup.

There are several computers, besides my PDA and the 'server' in the network. All the computers are running windows xp and serve different members of my staff--and all of which are putting in information pertinent to their job. The PDA is an HP IPAQ running Windows CE Mobile, second edition.

The goal in all this is for the PDA to recieve data in real-time from the 'server' (itself a windows xp machine) and for the PDA to immediately update the 'server' via wireless 802.11b.

I purchased Visual CE to write the forms for the PDA and their sister product, mEnable, to transmit the data between the PDA and the 'Server'. On the server is a TSR version of mEnable which uses ODBC to read/write to the vfp tables. (The ODBC driver being supplied by Visual CE. mEnable does not seem to be able to read the database containers, so I don't think I can get it to see views.)

I have no access to the mEnable code--everything else I've written from the ground up--so all is changeable.

My basic starting scheme was to keep things as simple as possible. Therefore, initially I kept all the tables as unbuffered and free. Doing it this way led to some interesting problems.

Problem: Sending records from the PDA to the Server resulted in dropped records. However, sending records from the Server to the PDA is 100% accurate.

I.e., New or changed records on the PDA were not being picked up by VFP.

Now, according to tek support at Visual CE, this is 'not possible', but perhaps there was an 'indexing' problem on the shared file and the 'vfp not being a true database server' maybe in some way altering the file.

I guess it's possible. My testing showed that:
1. VFP is so fast that it can completely read/write/update all the computers on the network in under 0.03 seconds, and
2. VFP was accessing the files durring an mEnable session.

My conclusion is that mEnable is using a record locking scheme. Also, once mEnable has 'synced' it assumes that the recieving side of mEnable got the information. Hence, if vfp finished before mEnable finished sending the record, nobody on the server will have the record, even though the PDA has the correct version. i.e., it can not be resent without rewriting the entire record on the PDA and starting over.

I then rewrote the whole thing to use optimistic table buffering on the vfp side of the tables. I don't know how mEnable views the tables..

Mikes' idea of forcing mEnable to use a view maybe the way to go, but I don't think that will avoid the problem of a partially written record from the PDA and nor does mEnable use the database container--so I don't know how to convince it to use a view. The transmission time of the PDA is easily measured in seconds, and for either VFP or mEnable to recieve partial records is useless. Hence, I then tried the starting scheme of this thread of letting my server startup by using the files in unbuffered shared mode, then durring read/writes to the pda files, forcing them into exclusive mode.

Problem: After opening the files in shared mode, if mEnable is allowed to sync, VFP can no longer reopen (after closing them) in Exclusive mode for the read/writes.

Further testing, however, showed that even with mEnable running and syncing, if I completely exited VFP back to the windows desktop and immediately reentered VFP (making no other changes), those same files that vfp could not open exclusively could now be opened exclusivly--but doing this every time the table needs to be polled is dumn and impossible.

Hence, my conclusion, is that it is not mEnable but rather VFP (or me) that is having the problem. It's almost as if once vfp fails, it stops trying--and no amount of Use..Shared, followed by Use...Exclusive will work.

In my most recent reiteration, I'm again rewritting this thing by :
1. putting all the 'sync' tables into a database containter;
2. changing the cursors to optimistic table buffering,
and ;
3. read/writing to the base tables by way of a transaction.

My hope is that if mEnable is read/writing to the tables at the same time as the transaction, that the transaction will fail allowing the server to backup and wait for mEnable to finish before trying again. I have no idea if this will work. I'm trying to preserve whatever table indexs mEnable is using..i.e., avoid writing to the tables by vfp while mEnable is writing to the tables.

Buf if mEnable is using record locks and is much, much, much slower then VFP, could VFP still not be in and out of the table before mEnable is finished? I guess the question is, does a record lock by mEnable stop the transaction? And what would happen if both vfp and menable are adding a new record at the same time?

As far as I know, I have no way of detecting when mEnable is 'syncing' from the desktop.

Anyway, I hope I've given you'all the picture. Overall, its been a networking nightmare that shouldn't have been...

If this fails, I guess I'll move on to views, but I don't see how that avoids the basic problem of possibly corrupting mEnable's index (if there are any) while mEnable is syncing.

Thank you'all very very much for your time, patience, and ideas--I'm sure glad to hear from you. I promise not to be so long winded next time. This is very frustrating.
Sincerely,
Alan

 

[DSummZZZ]

Hmm. Quite the dilemma.
Are there any sort of commit or buffering settings on the ODBC driver you could twiddle with?
Is it doable to have the PDA(s) connect to a temp table and have VFP periodically import the records from the temp table(s)?
Have you tried using pessimistic table buffering?


-Dave Summers-

Even more Fox stuff at:

http://www.davesummers.net/foxprolinks.htm

 

[alan232]

Hi Dave;

Thanks for reading my problem..
Anyway, by accessing the ODBC pannel under the windows control pannel, all I seem to get is '..this driver is licensed for Visual CE use only..' notification--no buffer settings that I can find...
In reading through the Visual CE notes, on a non-related issue, they suggest giving the PDA direct access to the base tables, but forcing the '...database appliaction...' to use 'views'. Their specific suggestion was intended for Microsoft Access, so I'm guessing it would apply to vfp as well...
However, I still don't know how or if that would solve the problem of partial read/writes of the PDA data by VFP. A quick calculation shows that vfp is running at least 1000 times faster then PDA to Desktop communication, and since mEnable is not stopping read/writes by vfp while it is syncing, I'm not sure what kind of result table I'm going to get...
My next stop is to try your suggestion of pessimistic table buffering...and I guess if that fails, reduce down to pessimistic record locks...
I'm hoping to find something that will work!

What I can't understand is that why would the writers of mEnable not build in some kind of check system that would allow the sync process to verify records were recieved correctly, or, failing this, simply resend the records? Apparently there is no such check.

Anyway, it looks like a long weekend ahead...I'll keep you posted,
Thanks Again,
Alan
(I'm starting to wonder how hard would it be to write my own communication package...I don't relish the idea...but I've got to make this work!)

 

[DSummZZZ]

Have you tried using FLUSH?
You may even need to try using the API function FlushFileBuffers(). Make sure they are written to disk before VFP tries to send any updates.


-Dave Summers-

Even more Fox stuff at:

http://www.davesummers.net/foxprolinks.htm

 

[jrumbaug]

Alan232

I'm trying to grasp your problem. You need to lock out mEnable while VFP does read/writes. But you don't know when mEnable is going to do a read/write. You said flock() does not prevent mEnable from reading and writing, can you confirm that USE EXCLUSIVE by VFP stops mEnable?

Jim Rumbaugh

 

[alan232]

Hi Dave and Jim:

That API call may very well solve the problem..excepting what happens if mEnable is writing at the time I call it?
And I don't know how to run it? How do I call an api function from within code?

Jim..
In a nut shell here is the problem:
VFP reads a set of common files (twelve of them to be exact) by a timer set at fifteen seconds. Hence, regardless of whatever the person at my windows 'server' is doing, the PDA files are being read/written to every fifteen seconds. I set up two job ques to handle input from multiple different computers. On the fifteen second interval, my 'server' machine reads on the files from all the computers and writes that data to the pda files.
On the other hand, the PDA itself has random input from me which is then syncronized at random time intervals by mEnable. The mEnable Server is advertised essentially to read/write/update files wirelessly between the pda and the desktop.
The scenario that is happening way to much (and once is to much for me), is that mEnable is 'syncing' the PDA with the Desktop at the same time or just before VFP goes into its read/write sequence. Some of the files being synced have multiple records that must be used (and printed) as a group. While I think I've got the read mechanism worked out for vfp to re-read a file when necessary, it appearts that once the pda THINKS is has synced with the desktop, it does not resend or update itself. So teck support is suggesting that the read/write by vfp is in someway messing up the indices used by mEnable. (I doubt this, but since I don't have the code to mEnable--or the rights to tamper with it, I must play along).
Hence the bottom line is this:
I need to stop vfp from read/writing to the files
while mEnable is syncing with them..and visa versa. The problem here is that it seems mEnable is using record locks and running so slow that vfp can read/write/update all the records on all the tables IN BETWEEN the time it takes mEnable to do just one record. Since it appears that mEnable is exiting between records, vfp can't seem to detect when mEnable is executing.

An encouraging note, in my most recent interation, I put the twelve tables into their own database container allowing my to use transactions. In actual testing, I noted that the transaction mechanisms to flash "attempting to lock..." while mEnable was syncing...if I did things right, this should allow vfp to do a rollback and start over...I hope.

Thanks again, any help is appreciated!
Alan

 

[alan232]

Hi Jim:

Addendum note;
When vfp has the files as exclusive, mEnable is stopped from reading them. (a good thing). But once mEnable has been allowed to sync, I can no longer force the files into exclusive mode for vfp without first completely exiting the vfp window back to the windows desktop and then reentering vfp. (I don't know why).

I hope I've answered your question.

Thanks again,
Alan

 

[alan232]

Hi Dave;

I'm not at all familiar with flush() and my documentation only mentions it.

What is the syntax? Does it have a return value for success or failure? i.e., instead of a transaction scheme, could I simply do something like:
if flush('umeds.dbf')
... read/write to umeds.dbf
tableupdate(.t.,.t.,'umeds')
else
let mEnable finish...try again later
endif

i.e., would flush() detect useage by mEnable?

My current scheme is:
begin transaction
try to update all the tables. set error flag
if all the tables can't be updated
if no error,
end transaction
else
rollback... and try again in fifteen seconds
endif

I'm hopefull...
Thanks for your help,
Alan