Close an Open File, Not a Table 1

[stanlyn]

Hi,

Assume that I opened a file with =FOPEN('d:\test.jpg',0)
Notice that I did not assign a variable to get the file's handle.

What commands could I use to close the file without knowing the handle?

Is there any commands where I can pass the filename in and close it?

I know the file is open by vfp as Win8 reports it as in use by vfp9 when I try to delete it at the OS level, and I'm looking for a way to close it, release it, unlock it from within vfp. If I close and restart vfp, the file can be deleted.

I've googled all over and cannot find any resource that shows how to close it, and many, many ways to test if it is open, but no way to close it if it's open.

Is there way of getting the handle to the open file?

How big is an average size handle, more than 2 digits?

Would something like this work?

x=1
do while x < 100
fclose(x) &&spinning through numbers 1-99
x=x+1
endo

What effects should I see with buffered vs unbuffered mode options for the fopen() function? Where is that discussed in detail?

Thanks,
Stanley

 

[stanlyn]

>> if you wipe files with SDELETE, which really overwrites the original clusters, your routine doesn't show that,

It doesn't show that because it happens after the encryption process has completed for ALL files. The initial encryption happens on all files with this implementation and after it completes, I run the sdelete command. At the finish of sdelete the harddrive should be completely protected because all files we want encrypted will be encrypted and any remaining free space will be zeroed. Or, am I missing something here?

Any other efforts to recover the data from unauthorized people would be a very time intensive operation. If they were to do screen swipes, they would have the task of naming the files and pointing them to the correct folder, and doing this on 380,000+ files would take a very very long time.

I didn't mention this before, because it was irrelevant to the question, but this new implementation has a new "call home" feature that also prevents access if their account if not in good standing. I could easily hook this encryption process to make sure they are coming from where they are supposed to be coming from. Such as... detect what wan ip they are coming from and enable/disable the decrypt routines based on that, as that would stop any decryption if machine is at any other wan connection, as thew viewers would stop viewing because the decryption was disabled. (All of our customers use static ips. Therefore if the whole machine gets stolen and even if they have all the login accounts and passwords, the data still could not be breached.


>> measurements you should do are outlined in a very broad way

And that is why we are doing this. We need protected on all these fronts, so we are stepping up...

Thanks Olaf,
Stanley

 

[Olaf Doschke]

> At the finish of sdelete the harddrive should be completely protected because all files we want encrypted will be encrypted and any remaining free space will be zeroed. Or, am I missing something here?

Well, from the description of SDELETE covering all unused space is one mode, obviously not very performant, another is processing single files (while they are there, of course), and that would go instead of Delete File (gcFileName), which you do before you reuse the filename to hold the encrypted data.

Quoted from

https://technet.microsoft.com/en-us/sysinternals/bb897443.aspx

How SDelete Works

Securely deleting a file that has no special attributes is relatively straight-forward: the secure delete program simply overwrites the file with the secure delete pattern.

It goes on talking about sepcial file types and sparse files and how trick it is to get at the sectors via Defragmentation API. But SDELETE is not only capable to overwrite all unus4ed space, but to overwrite the existing file sectors you only need to invalidate this way before recreating the file with its encrypted version. The SDELETE call you do might run in a background process, but will have lots to do about all unused space. What's also weak about processing unused space is, if used space only was overwritten once the unencrypted parts of previously stored files of used sectors might still be read from the hdd platter, if data theft would go into that detail. Maybe petient data rarley is worth that effort, but acting on the sectors you definitely want overwritten several times with a secure deletion pattern should be done instead of processing free space as a post process.

Bye, Olaf.

 

[stanlyn]

Olaf, there is a -z flag that specifically zeros free space. The command is "sdelete -p 2 -q -s -z e:" which does 3 passes, recurses directories, quietly, and zeros all free space on the e: drive.

Maybe running this as a 2 pass command before encryption and another 2 pass after the encryption. That should more that strong enough. I'm testing it now and will attempt to find deleted files with "emergency undelete". Do you know of any better tools that can be used in this test?

Performance is not a big factor here as this only runs once, which is when we update their system with these features. After that, there will never be more than one image file on the drive in an unencrypted state. It is deleted, and I could run the sdelete on the single file??? I've never tried sdelete on single files before, but I'll try it.

Stanley

 

[Olaf Doschke]

I understand that SDELETE has two options. You still don't understand, what you're doing and how it fails to be secure about the non free disc space.

SDELETE has mainly two options: 1. overwrite an existing file, 2. overwrite free disc space. You only seem to use option 2. You think it's sufficient, because all used disc space has it's encrypted data and so data is overall safe. But physical data rescue can also restore overwritten block. The security of SDELETE is strongly depending on overwriting sectors several times, a single overwrite does not clear the prevous magnetization completely enough.

As you don't overwrite the sectors of unencrypted files, but delete them normally and then later care for these freed sectors with SDELETE -z option, you think that's good enough. But potetntially all the sectors of the encrypted files are only overwritten once with encrypted data and therefore have never been overwritten with the patterns that clear previous magnetization better than anything else and only overwritten once. All allocated and used file sector potentially contain unencrypted data. Data you can only get at when having the physical hdd at hand, but still you can get at them.

If you have an unencrypted file on disc and you want it's sectors to be securely delted, you can't just hope the freeing of the sectors will later be handled by the single SDELETE of free space. Part of the sectors will not be used by overwriting, but part of them will be reused. To really have control about the secure deletion of a files sectors you have to use SDELETE to overwrite the file as long as it exsits. So you 1) encrypt the file and store that separete (as you do) then 2) SDELETE the original and then 3) reuse the file name for the encrypted file.

You are relying on freed space not being reused in the same pass of encrypting files, and that's what's wrong.

Whats okay with your solution is that it's covering all hdd sectors, the ones used are used by encrypted files (and maybe other files non important for data protection concerns, but that just aside) and free sectors taken care of by SDELETE. That's fine for any normal software read of the hdd sector wise it won't find any unencrypted data, but getting at the hdd platters you can see the previous magnetization on many sectors. Physically the information stored by magnetiztion is not simply perfectly 1 and 0, that's why you can see whether a 1 overwrote a previous 0 or 1 or a 0 overwrote a previous 0 or 1. The overwrite patterns SDELETE use are levelling out magentization often enough, so you can't see it anymore, the next overwrite would just enable to see the overwrite pattern, which is useless information, of course, that's how it works.

If your hdd is SSD you actually wouldnÄt have the problem, indeed SDELETE would just cause the SSD life to shorten as it wears the memory cells. You don't have the magnetic hystersis in that case.

Bye, Olaf.

 

[stanlyn]

Olaf, so you are concerned with sectors that have not been completely overwritten because the file size was too small to fill it? We know the smallest units are sectors and a file is written on sectors filling each until its all written and can span many sectors. Its that last sector that is not completely written. So, is it this sector that you are concerned with? As it is really a fragment and more than likely useless on its own, except file really small files.

>> So you 1) encrypt the file and store that separete (as you do) then 2) SDELETE the original
>> and then 3) reuse the file name for the encrypted file.

Wouldn't we be losing the file's meta data? As any delete and create would cause the new file to take on totally different meta. You are the one that pointed that out, and I agree...

So, would the sdelete process go like something like this:
1. do as I do now by encrypting the file in-place preserving its meta data,
2. copy the file somewhere?
3. sdelete the original?
4. copy the file back and renaming it to its original name?
5. sdelete the first copy?

What happens to the original file's meta data? Wouldn't that have to be dealt with?

Would we be better off if 1) we encrypt the whole drive from scratch with no files on it, 2) reload all the files from backups preserving the meta data, and 3) apply my encryption to them? This way, there would never be a time where unencrypted files exist on the hdd. Is this approach secure enough?

Thanks,
Stanley

 

[Olaf Doschke]

>Wouldn't we be losing the file's meta data? As any delete and create would cause the new file to take on totally different meta. You are the one that pointed that out, and I agree...
Well, but you still do DELETE FILE and so you still create a new file and have to copy meta data about it. It's the only chance to work on the sectors of the original unencrypted data. It's a pity sdelete.exe doesn't come with an option to just overwrite the content without finally deleting the file and its name from the filesystem TOC. I just downloaded it from

https://technet.microsoft.com/de-de/sysinternals/bb897443.aspx

and tried sdelete -p 3 testfile.txt and the file is finally deleted.

You're right this is really a paranoid prespective, if someone really took the hdd hardware and analyize it with data recovery methods acting on the physical platters.

Let's assume a simple case:
Initial situation: The disc has just 5 sectors, 1 unencrypted file covers sector 1, nothing more on disc

Processing the file you first create the encrypted file, let's assume it's saved on sector 2.
Now you delete the original file, freeing sector 1.
Now you copy the encrypted file recycling the file name (and other attributes) and it could land on any sector but 2. In my worst case scenario it is stored on sector 1, though I suggested earlier the OS would rather only overwrite used sectors when there are no unused available anymore. Still it's possible.
Now you don't need the encrypted temp file anymore, you delete it, too, and sector 2 is freed.
Finally you SDELETE the free space, sectors 2-5 are overwritten two or three times with a secure deletion pattern or zeroed.

With software you'd read rubbish or just zeros from sectors 2-5, which is fine, also sector 1 now contains the encrytped data and the unencrypted data was overwritten by it. The big point is, it was only overwritten once and the original data was on it before. With the physical platter you can restore the unencrypted data previously written on sector 1.

In the average case indeed you would only find partial file blocks this way, but a simple scan of all hdd sectors for the most probable previous bits can reveal partial unencrypted data. There are devices data recovery companies use, that take any typical platter and instead of reading it binary they rather read the finer analog magnetization and the old bits resemble in peaks besides the new peaks, for example. You can read the current and previous state of sectors quite well.

Bye, Olaf.