find non vfp file open in a non vfp application

[AlliMac]

In thread 184-1674267 the poster asked how to tell if a file was open. After a bit of prompting he/she explained that it was an Excel file that was "open, meaning visible in your taskbar".

Mike Lewis offered a solution using APIs, GriffMG offered a solution using low level file calls and there was other advice.

I don't know what I'm doing wrong but the low level file calls don't work for me as hopefully is clear from the following:

With the VFP IDE open, use file explorer (or whatever) to open, say "C:\MyFolder\MyFile.txt", a file with some text in it.
Return to the VFP IDE and enter the following (and get the following results)
public gnHandle
public gnBytes
m.gnHandle = FOPEN("C:\MyFolder\MyFile.txt", 12)
?m.gnHandle && 21 (For example)
m.gnBytes = FWRITE(m.gnHandle, "This is the house")
?m.gnBytes && 17
?FCLOSE(m.gnHandle) && .T.

Switch to the open file "C:\MyFolder\MyFile.txt" - there is no change.
Either: close the file without editing it, open it again and there is the "This is the house"
or: edit the file, close and save and then reopen it and there is no "This is the house"

I don't understand why what I get seems to be so much at variance with the advice given, especially from Olaf Doschke, a man with whom I would be very afraid to start an argument. From what I read, I expected the FOPEN() funtion to return -1 if the file "C:\MyFolder\MyFile.txt" was open in another application

Mike Lewis's API solution worked for me, but I also need to know the path name for the file. My knowledge of API usage is very limited, I use them strictly on a "monkey see, monkey do" basis.
However I found 3 possibilities, GetFinalPathNameByHandle, GetFinalPathNameByHandleA and GetFinalPathNameByHandleW. No idea what the difference is. I tried them inside Mike's code. My declaration for all of them was
DECLARE integer GetFinalPathNameByHandle IN WIN32API ;
integer hFile, string lpszFilePath, integer cchFilePath, ;
integer dwFlags
Same for the ...A and the ...W

My usage was
lcBuffer = REPLICATE(CHR(0), 250)
GetFinalPathNameByHandle(hNext, lcBuffer, LEN(lcBuffer), 0x8)
Same for the ...A or ...W. hNext is the handle returned for the file in Mike's code.
This failed. I managed to paste in GetLastError() code from an ancient VFP knowledgebase article. This told me the error was 6 "The handle is invalid"

Could someone please tell me (1) what I'm doing wrong with the low level functions, (2) what I'm doing wrong with the API functions and (3) How to find out if a non VFP file is open in a non VFP application, and if so its path.

If I haven't yet bored you all to death, the practicalities of my question are as follows: My (tiny) application frequently writes to a file named user.log as a means of maintaining an audit trail. The user can view and edit the log from a shellexecute in the application menu. I want to cover the situation where the user (or for that matter anybody else with access to the folder) may have opened the log and perhaps forgotten about it. I want to be sure it is closed in any other application before it is written to by mine. If it is open, perhaps being edited because it has become to big, then tough, out it goes. If there is another application with a user.log and it is open, I want to leave that one alone.

Thanks for your patience

 

[Mike Lewis]

Regarding "chopping off the top to bring it to size", this is something that can, and should, be done within the application. If you decide to use a DBF, then your program could check the size periodically. As it approaches, say, 1.8 GB, it should delete the oldest. say, 0.5 GB of records, and then do a Pack. You would have to do this when no other part of the system can access the log, as it requires exclusive use.

This is what I do in the application I mentioned above, and it has been working well for over 20 years.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

Txt logging is not at all unusual, the most used VFP function for it is STRTOFILE, not FOPEN/FRWITE/FPUTS/FCLOSE. It has the same problems, though, with user editing of logs.

You say you (and your users) main intent is

to chop the top off it to bring it down to size

Why not do as I suggested early on? You can determine the size of the log file and if it exceeds say 1MB (not even 1GB) create a new one with a serial number. Old ones could be archived or erased. That makes editing for that reason completely unnecessary.

And you also said

I was really thinking of inadvertent editing or access.

Well, is it inadvertent, if you actually offer the feature to open it by shellExecute from your application? How does it make that inadvertant?
Usually you log errors in such a log so you as developer can look at it, the user has a) a messagebox of an error, and b) usually no idea what to make from the error nor from a list of errors in a log.
There are other logs, maybe you log when the application start and ends, so a mismatch of a start without an end log row means a crash of the application. Such things are quite usual.

The more meaning your log entries have, the more likely you rather want a data file to store what you log and more than just a text field to store the lines of text, to be able to filter better. That a dbf physically in chronological order does only mean that, not that you easily can filter for all records from June 2022, that's simple with a date field, though.

You can also use a dbf you actually initialize with empty records up to the maximum size you want to allow it to grow to and then sort it by a date field. When adding a new log entry you replace data in the record at the top in datetime order, which is the oldest record. Important is you replace the date fielld that sorts it to the current datetime and thus make it the lat record. Keep that order by a datetime filed and you always replace the oldest record with the latest log data and have a fixed size log, where the chronological end is somewhere in the DBF, not at the physical end of file.

Thare are so many ideas, you just have to thnk a bit harder about your wanted features and then work towards them.


Chriss

 

[AlliMac]

Thanks Mike for your latest post. When I change the log to a dbf I certainly intend to monitor its size within the application. I already have two ideas for that to think about, yours and Chriss's (see below)

Thanks Chriss for your latest response. If I were to stick with a text file for logging, your suggestion is certainly a much better way of limiting the size of the file (if in fact it ever got to be big enough to be a problem). However you have more than convinced me that a dbf is a better solution for logging.

By inadvertent I mean the user might open the log to see what he/she did on such and such a date or on what application record, forget about it and do some editing which results in a log entry. Perhaps inadvertent was not the right word. I also take your point about date and time. Changing to a dbf requires a lot more thinking and I'll be reading this post over and over again to help me.

Thanks again

 

[Mike Lewis]

the user might open the log to see what he/she did on such and such a date or on what application record, forget about it and do some editing

It's hard to imagine a situation where the user is able to edit a log file. Surely, the purpose of a log file is to provide a record of the things that actually happen within the application. Why should a user be allowed to change that record?

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[GriffMG]

Footprints in the digital snow...

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

I'm trying to cut down on the use of shrieks (exclamation marks), I'm told they are !good for you.

There is no place like G28 X0 Y0 Z0
​

 

[Chris Miller]

By the way, logs can by error contain information you wouldn't want to leak, like credentials. Such things already have made the news, and not just once. As the major point was shortening the log, that's better made a task of your logging mechanism, and if it comes to important log information that can help you prove what has been done, legal rights of users like data privacy can play a role on the other side, i.e. some companies would get in trouble if you log something identifying the user and the time he did something. Even though on the other side there are tools in which employees are asked to report what they worked on in general.

On the other side log manipulation to change information after the fact can also bring you into trouble. And thus there are techniques that start with computing a simple hash for each log entry (which is not very safe, as it can be recomputed after changing logged information) and go up to cryptographically chaining the log entries, which makes detectable any manipulation. While the information that's removed then still is removed and you only have evidence of manipulation, you can, of course, put log data within a database that limits privileges to who can access it or log in two different locations, etc.

Then we don't talk of DBFs anymore, though a DBF file also can be secured by the file access control mechanisms of Windows/NTFS. The not-so-trivial part is that your application runs with the user's account and privileges and should allow writing the log under that condition, but there are ways to let your EXE impersonate another user - overall or just while logging - and there are ways with Windows cryptographic API that gives privileges to your process, not a user. That makes access privileges only coincide with the user as long as he uses your EXE, not as the user himself when he uses other Windows applications, tools - even hacker tools.

I won't go into details about all of this, but any intention of letting users get to the data they legally are allowed to know without enabling them to manipulate that is possible. I.e. all legal rights of both them and you can be fulfilled, if you just dig deep enough into it.

Just saying this, so you know this is still not at all the end of all thoughts you can put into anything, be it as simple as logging.

Chriss

 

[AlliMac]

Thanks Mike Lewis and Chriss for your latest responses. The whole logging thing clearly requires a total re-think and it will get that. Very interested in Chriss's remarks on data security and access. My application is probably to insignificant for this to be an issue but as I revise the logging system in general this whole thread will be my guide.

Thanks to everyone for your responses

 

[Chris Miller]

Thanks for that feedback.

In your case, if you don't need to worry much about the security of the log information - just want no errors during logging - the simplest solution would be to do as VFP does in coverage logging and keep an exclusive file handle all the time. So you get it at application start and don't close it before application end. It stops any parallel access. And you don't even need to worry about crashes. In that case, Windows will close file handles anyway.

You may provide a log view feature within your application, as it's the only process able to write and also read the log file while your application runs.

Of course, log access still is possible after your EXE quits. But if that's okay the exclusive log access during your application session fulfills its job to avoid the problems of concurrency in any of the forms I wrote about. Its intention is not to exclude the user totally from the log, just to avoid any concurrency problems while your application writes to the log.

To go that route of a solution you still could use the file functions fopen/fseek/fwrite/fclose. And, of course, you can use a DBF for logging and can USE yourlog.dbf EXCLUSIVE. The code is even clearer than fopen(file,12) is. Same strategy and reasoning. Including the same feature: In case of an EXE crash this exclusive dBF file access is closed by Windows, too.

Well, and in case USE yourlog.dbf EXCLUSIVE or fopen(logfile,12) fail at application start, you simply refuse to continue and quit.

Chriss

 

[Mike Lewis]

And, of course, you can use a DBF for logging and can USE yourlog.dbf EXCLUSIVE.

But what if you have multiple applications all using the same log file? Or multiple users of the same application? In that case, wouldn't it be better to use the table SHARED, and to apply record locking for the very short time that it takes to write to the file?

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

Mike, if you want to prevent any problem of concurrency, isn't it the logical consequence to use a file exclusive?
Even if you now have a central log file, which I doubt makes sense for text-based logging, then you can provide one log file per installation, using ID() within the name when storing it in a central share or storing it local.

Anyway, it is not for data you want to share with other users, it's simply the log data of one installation, one application instance, no more, no less, there is no need to share access to it. That was the root of problems with the log file, wasn't it? That opened in notepad, once you save you overwrite what was logged in the time of editing.

Of course, technically you can use a dbf shared and then both write and read shared. You you can make it a dump of log data of all application installations and instances, and hard to later interpret it, unless you add a field for it to filter for. But then I'd rather prefer to union data of multiple logs, if at all, than the need to filter out just the log entries of one installation. Usually, I look into one instance log data to identify a problem with that installation.

In the end, it depends what you like best and is the decision of AlliMac. In case of keeping it a text-based log file, I'd think of exclusive usage by FOPEN(file,12) or FOPEN(file,2) as a nice way to prevent any concurrency problems and suppress any parallel usage of the file with notepad or any other text editor. In the case of DBF the parallel use of text editors falls flat, that's true, so you can decide to use a shared log. But to see log data per instance separate log files or DBFs are still useful, they also grow less than if storing all log data into one dbf. If you want to do any overall statistics or find common problems in all data, you could append all DBFs into one for that processing.

Chriss

 

[AlliMac]

Thanks for the latest posts Mike Lewis and Chriss. More food for thought. At present each user has his/her own user log, but when this thread started me thinking more deeply, I wondered if there shouldn't be one user log for the application. After all it is there primarily as an audit trail (although I'm not sure how often it has been used that way), eg did somebody actually send that job sheet by email to a customer who is now claiming he never received it? It might not matter so much who should have sent it, only if it was in fact sent.

However Chriss's latest post suggests a reason for not having a single user log, so back to the drawing board on that, although it's probably not an issue for my application.

Also I guess you could restrict viewing access to the manager or whoever. But my application doesn't have levels of user so that wouldn't work well. Possibly the easiest way of dealing with it in my application would be to have a common log viewable but not editable by all users but with the application in single user mode.

So far the most important issue seems to be control of the log or logs by the application and it seems that the best way of doing that is to keep them in tables. More thinking required for all the rest.

Thanks again

 

[Chris Miller]

It might not matter so much who should have sent it, only if it was in fact sent.

That is speaking for a single shared log. the only reason still speaking against a shared DBF is when you absolutely want to have full control and log without any interference from editing, inadvertent or not. The exclusive access prevents that, but also prevents shared logging.

If you don't fear users trying to look into a DBF with any tool that might not open it shared only, then you may use a shared central DBF, Mike is right about that. At least a DBF does not incentivize users to edit the file with notepad like a txt or log file does. Simply because the file extension .dbf will only be associated with FoxPro on a developer PC, it has no association on a normal user PC, even with a Foxpro application and runtimes installed.

A step beyond that would be storing log data into a SQL Server database table, that's when users will be cut off of finding log data in a single file they can easily get at. It's over the top to set up an MS SQL Server, even just the Express version of it, just to have a log table in there. Too much effort also because in that specific case there is more direct evidence in the outbox of a mail software than in log data. No doubt a log can be your starting point of researching what was done, not only in the case of mails. And for that something users can't manipulate to fake to have done something or hide an error by deleting log entries may be worth using a database server. It's also not that far fetched, if there is a database server available anyway.

Chriss

 

[AlliMac]

Thanks for your latest response Chriss. I can see that in many applications access to (any) data would need serious control. But my tiny application doesn't warrant that depth. Anyone who uses it (it is supposed to be multi user but I'm not even sure that there are multiple users) is equally interested in data entered by all users and can also edit data entered by other users (such changes would be recorded in the user log of course).

From what I've learned from this thread, a fixed length dbf, viewable but not editable, is probably a much better way of maintaining a user log than my text file method. But this has become a long thread and there is a lot in it to think about. I'll be reading it many times yet.

Thanks everyone.