long vs short filenames

[Thai Guy]

Hello! I've just arrived and may not have selected the proper forum for my post. If it would be better located elsewhere, please move it and let me know.

My concern is that I am puzzled by how FoxPro 2.6 is unable to use long filenames and instead uses something like this: "palacc~1" rather than the full wording, i.e. "palaccount" . Could someone provide an explanation for this behavior? Thx in advance. THAI Guy in Thailand

 

[Mike Lewis]

Welcome to the forum. We deal with all versions of FoxPro here, so you are in the right place.

The reason that you are seeing the short filename is that FoxPro 2.6 was written to run under MS-DOS and Windows 3.x Those versions did not support long filenames, so FoxPro 2.6 is unable to show them.

That doesn't mean that you have lost the long filename. It is still present in the file, and you should see it if you open the file in VFP or any other 32- or 64-bit application.

Mike

 

[mmerlinn]

My concern is that I am puzzled by how FoxPro 2.6 is unable to use long filenames and instead uses something like this: "palacc~1" rather than the full wording, i.e. "palaccount" .

It seems to me, although I never use them, that FoxPro 2.6 for Mac CAN use long file names. However, as Mike says, neither DOS nor Windows versions of FoxPro have that capability due to built in limitations of those platforms. I have no idea whether or not FoxPro for Unix can.

 

[GriffMG]

am i right in thinking, that short file names have to change when there are potential conflicts

 

[mmerlinn]

am i right in thinking, that short file names have to change when there are potential conflicts

Yes.

If you have "customerzip" and "customername", FoxPro substitutes "custom~1" and "custom~2" to avoid conflicts.

 

[GriffMG]

and from drive to drive, folder to folder. custom~2 might not work - good in the fs, no good if you saved ~2 in your db

 

[mmerlinn]

and from drive to drive, folder to folder. custom~2 might not work - good in the fs, no good if you saved ~2 in your db

Exactly why I do not use long file names.

 

[Chris Miller]

am i right in thinking, that short file names have to change when there are potential conflicts

Yes and no. When a file name is too long it gets shortened to 6 letters plus ~1 and if a second file has the same six first letters it gets the suffix ~2. Those stay, they don't get renubered everytime a new file with the same six letters prefix is created-

Taken the example of mmerlin with customerzip and customername, when you move them to another folder where customerstatus already has the customer~1 short name, the customerzip and customername get new suffixes. But if you keep files where they are and delete one, that doesn't mean suffixes will get renumbered, names stay fixed.

What can get you in the situation of a deleted file besides the obvious problem your data still refers to a deleted file is that a third new file then can get the freed short name. though it's not the same as the deleted file.

So say you had
customerzip.txt customer~1.txt
customername.txt customer~2.txt

Delete customer~2.txt and instead create customerhome.txt, then that will become customer~2.txt And notice: if you instead delete custmer~1.txt aka customerzip.txt and create a new customerwhatever.txt, that still doesn't get ~3 but will fill the gap and become customer~1.txt But the kept file in both cases keeps its short name, the FS does not renumber all short names just because one file is deleted. The DOS name isn't just a label VFP generates, it's part of the actual file even in an NTFS formatted drive.

It's a matter of how you deal with this when a file is deleted in the first step, i.e. if it's not just a user deleting files your application needs, which is a problem in itself, no matter if you can only refer to a file with short or long name, but the file is deleted by your software intentionally, then also delete the record referring to it at that same time and when your software creates a new file name also store its short DOS name, you'll be fine throughout all lifecycles of all files. So you can work with long file names and their short DOS names, if you work that out correctly.

It's hassle free, of course, if you limit the file names to short names.

About the same can be said about short field names that refer to a dbf with long field names. Usually, you'll have more problems because FP 2.6 won't handle all DBFs due to data types not available and other discrepancies of the DBF file format, it doesn't help that a DBC table still has short field names shortened by the same principle as file names, just with a limit of 10 characters, instead of 8.

When you have to deal with files existing, you have to deal with their DOS names as you can only refer to them by short names. But they don't change file names without changes in the file system as discussed above, though. Which always meant another new file got the old short name, recycled, as the FS doesn't keep track of a counter it always increases. The problem never is that an existing file gets renamed in its short name. Once the short name is given, it's fixed.

And btw, you can see short file names also in VFP versions with a flag of the ADIR function you can set to get a list of short (DOS) names instead of long names, so you can see what legacy FP versions see with VFP9, too.

 

[Chris Miller]

It's even mildly amusing how short file names can affect name conflicts, completely independant of FoxPro.
Open up Windows Explorer and create a new empty directory.
In that directory first create a file named 123456789.txt
Then try to create a file names 123456~1,txt (not by means of creating yet another long file name, but explicitly trying to give it that DOS/short name yourself).

This is what you get:

As 123456789.txt already uses the short name 123456~1.txt and that has to uniquely identify that file, there can't be another file named 123456~1.txt, even though the 123456~1.txt file actually has the long name 123456789.txt

What's mildly amusing is that Windows Explorer does not suggest the file name 123456~1 (2).txt, but 123456789 (2).txt
What you can learn from this exercise is that short names are already baked into Windows/the file system, not something FoxPro uses internally, only, and that due to different name conflict strategies used for short/DOS names and double file names in Windows Explorer, the suggested name alternative differs from what you'd expect. Due to finding that 123456~1.txt is the short name of the 123456789.txt file, Windows Explorer does suggest 123456789 (2).txt on the basis of that long name, not on the bais of the short name.

And the final lesson to learn about this is that hassle free file names for FP 2.6 not only need to be short, they should also not be names to which long names could be shortened, so one simple rule you can define for your file names is not include tilde (~) characters in file names yourself, so they only occur in the DOS/short names.

 

[Joe Crescenzi]

To be clear, FoxPro does not create the shorter filenames, it's Windows.

For those old enough to remember, Windows originally was just a program that you launched from DOS, by typing WIN. Back then, you could exit Windows, and you were back at DOS because it wasn't a true Operating System, just another application.

Anyhow, DOS only supported 8 character filenames with up to 3 character extensions. When Microsoft eventually adopted long file names, they still stored them in the 8.3 format in the FAT table and stored the longer name in a reserved area outside the normal space, so any program that requested a file could still see it.

As filesystem generations evolved (FAT16, FAT32, eXFAT, NTFS, etc.), they added better support for longer filenames, but still made them backwards compatible so older applications could still request any filename and still find it.

SO... Even to this day, the job of creating those shorter versions of filenames is still done in the operating system, so they're entirely out of the control of individual programs like FoxPro.

As a side effect, you will never know for sure what the 8.3 version of a long file name will be when it's created, copied, or moved to a directory. So, if you run a program that doesn't have native support for long filenames, you should avoid referring to anything important by filename. So when I still supported my DOS users, I never wrote PRG files with long names, because I didn't want to risk calling the wrong one. There is no way to know for sure what 8.3 name Windows will give MyMenuProgram.prg and MyMenuProgram2.prg.

 

[Chris Miller]

There is no way to know for sure what 8.3 name Windows will give MyMenuProgram.prg and MyMenuProgram2.prg

That's true.

If you determine the short name of na long file name that you just created yourself somehow - I don't think that is available in legacy FP, though - then you could store that and refer to it, as short file names don't change.

I already said that and recommend using short names for your own file creation, if you have to support files which already exist from other work/applications, you can rely on short names not varying, though, unless the other work/application is deleting, creating files and reuses long names, that don't necessarily get shortened the same way.

For example, in the demonstration I made proving the Source of short file names is Windows itseld, you couldn't create a 123456~1,txt when first creating a 123456789.txt file that gets this short name. But inversing the order you can write a 123456~1,txt file and then a 123456789.txt file will get the short name 123456~2,txt Which means you can't determine the short name knowing the long name, you explicitly have to determine the short name of a file from Windows and the file system somehow.

In VFP9 that's possible by ADIR, as I said, I'm not sure you get it from a legacy FP function. So while FP only sees files by their short name, you have a problem of knowing which short name belongs to which long name, if you have short names that start the same and have the ~1, ~2, etc suffixes. That's a problem to solve in the case you're confronted with existing files you have to work with as you can hardly dictate for the other work/application to change to only creating/maintaining files with short names.

But for your own work with FP 2.6 just restrict yourself to short file names and you're out of trouble.

 

[Joe Crescenzi]

That's true.

If you determine the short name of na long file name that you just created yourself somehow - I don't think that is available in legacy FP, though - then you could store that and refer to it, as short file names don't change.

Yes... but only if the contents of the directory are guaranteed to be static.

Windows creates the 8.3 filename once a file enters a directory and it definitely won't change it unless the files are copied to another location that potentially has existing files with the same 8.3 filename as it's root.

For example, a file MyLongFilenName.XYZ, copied to a directory where there are already 4 similar 8.3 names (~1, ~2, etc.) and this particular one is ~4 will likely become ~5 regardless of what the original 8.3 name was.

Another example is what happens with edits. When you open a file in some programs, the original file may be copied or renamed as a .BAK. If that's the first version of that .BAK version, it's possible, the .BAK version will become the new ~1.BAK version of the 8.3 name. After saving the file, the program may re-write the file and it could become a new ~5 filename.

Basically, if a file has the potential to be copied or moved to other folders with existing files, or edited by programs that create temp files and re-save the file, you have the potential to point to the wrong 8.3 name.

 

[Chris Miller]

The point I didn't get through, I think, but still want to stress is:

When you're confronted with files as they are and can't change the outset, not everything is lost and without any chances for a legacy FP application to refer to files by name, also short name. You don't always have to expect things that effectively lead to files being replaced by new files with the old long name, but different short name.

If you're confronted with situations like Joe described with editing via rename original to bak file and writing the modified file version with the old long name but getting a new short name, you will have to deal with the problem and better have the long name available that's not having the same risc.

Well, provided legacy FP only sees files by their short name and even calling shell commands like DIR won't show you the long names - or only the long names and no clue which short name correspsonds to them, then you still have Windows API to get the long filename from a short file name with the help of FoxTools and RegFn()/CallFn():

Set Library To Home()+"foxtools.fll" Additive

Local lnGetLongPathName, lcLongName
lnGetLongPathName = RegFn32("GetLongPathName", "C@CI", "I", "Kernel32")

lcLongName = Replicate(Chr(0),260)
lnLength = CallFn(lnGetLongPathName,"C:\Progra~2",@lcLongName,Len(lcLongName))
? lnLength
? Left(lcLongName, lnLength)

Not sure whether you have RegFn32 in FP 2.6, if not you should at least have RgFn within Foxtools.fll and that's available since at least FP 2.5, also see what the Hackers Guide has as example for legacy FP, especially:

FoxTools

An irreverent look at how Visual FoxPro really works. Tells you the inside scoop on every command, function, property, event and method of Visual FoxPro.

hackfox.github.io

The main idea of this is you could determine long file names and store them for a more stable reference, though you may not need this, if your files are static and only get added to, never erased and overwritten or moved (copied) to a new directory not in chronological order, getting new short names, especially if some short names have ~2 or higher suffixes. Which means a non problematic situation is where each short file name of a directory only has ~1 suffix, then coyping all files elsewhere won't change that, as each file name beginning is unique.

So to circle back to the point I want to make: You're not necessarily lost with only storing short file names for file referencing, it depends on the outset and if you're having everything in your own hands just avoid long names. If not, this API function of Windows can help you to determine the long file names as a more stable identifier of files.

Then you only have the same trouble you have, when files are erased or renamed or moved, which you also have with long file names, if they are not static. All in all there is no stable id for files. File systems refer to files by (full) file name and I guess depending on the file system you have an id like a file GUID, but that's also gone, if a file is not moved, but copied somewhere else, getting a new entry in the filesystem TOC (table of content) and therefore a new ID, even if the stem name is the same. So the long file name is probably the most stable identifier and reference to a file you can store in a table. Things like a cryptographic checksum would even change with any file change, even if it's still the same file, just modified, so they are even less relevant as file identifier. The most valueable identifer for a file is its long fullpath name.

The inverse function GetShortPathName also exists and is used the same way, I spare to give the counter example, so when storing long file names for stable reference to files, you can use GetShortPathName to detemine the (currently) short file names to access the files with FP 2.6

 

[Joe Crescenzi]

The point I didn't get through, I think, but still want to stress is:

When you're confronted with files as they are and can't change the outset, not everything is lost and without any chances for a legacy FP application to refer to files by name, also short name. You don't always have to expect things that effectively lead to files being replaced by new files with the old long name, but different short name.

If you're confronted with situations like Joe described with editing via rename original to bak file and writing the modified file version with the old long name but getting a new short name, you will have to deal with the problem and better have the long name available that's not having the same risc.

Well, provided legacy FP only sees files by their short name and even calling shell commands like DIR won't show you the long names - or only the long names and no clue which short name correspsonds to them, then you still have Windows API to get the long filename from a short file name with the help of FoxTools and RegFn()/CallFn():

Well said. The key phrase being "...as they are".

So the bottom line is if a program only needs to refer to files as they are right now, you can absolutely trust the short names.

However, any long term assumptions about the short names being the same years later are dependent on whether those files move or how they are edited, so if you store those short names in a database, you are not guaranteed they will always have the same short names.

 

[Chris Miller]

Well, lets take your example of editing a PRG and having a BAK, as VFP does it.
Before starting this code, ensure that editing a PRG creates a BAK. That's configured in options, tab IDE, Save options: Make backup
And then SET SAFETY ON so saving a modified PRG leads to the safety question whether you want to save changes, which the {ENTER} part of the KEYBOARD command will confirm, so you don't have to do anything but run this code:

Clear
Cd GetEnv("TEMP")
Erase longprogramname.*

StrToFile("","longprogramname.prg")
ADir(laShortname,"longprogramname.prg","",2)
? 'before editing:'
? laShortname[1,1]

Keyboard "* I was here{CTRL+F4}{ENTER}" plain clear
EditSource("longprogramname.prg")
Doevents

? 'after editing:'
For lnI = 1 To ADir(laShortname,"longprogramname.*","",2)
? laShortname[lnI,1]
EndFor lnI

Your assumption the BAK file would consume the short file name of the PRG and so the modified new PRG file would get a new short name is wrong, because files with different file extensions can all have the same short stem name.

So you actually have to come up with a better example, where that really does happen.

The warning is not without reason, that's true. But in the end also long file names may not exist anymore after years. If your data references files, that should always be done with the precaution files may have got lost due to users manipluatong directories manually anyway, no matter if that's tempering with short or long names or erases or replaces files. The use case for checksums obviously is very valid for detecting tempering with file content, if you want to ensure a DLL is still as you know it, for example, and stay away from tempered DLLs. But that's a different use case then reidentifying or reusing a file like an image file for UI purposes. But even if that images long and short file names have not changed, you may display a tempereed image. So even such a use of referring to files depends on what's important to you, That you find a file by that name (long or short) may also not be sufficient.

 

[Dan Olsson]

I guess I'm old-school I still only use 8.3 filenames and don't use Swedish diacritical letters either (ÅÄÖ)

 

[Chris Miller]

Dan, do you store files in system directories?

Even that also exposes you to the problems one might have with long file names as directory names are involved, too. For example the C:\Program Files\ directory or even just your user directory, if the Windows login name is longer than 8 characters, like OlssonDan already has 9.

As longwinded as the discussion is, though, we all can agree a legacy FoxPro application would best be organized and run with all file and path names being short. There still are possibilities to cope with long file names or system directories, too. If it only is, that you change directory and refer to files by the filename only, without the path part.

 

[Joe Crescenzi]

Well, lets take your example of editing a PRG and having a BAK, as VFP does it:

Your assumption the BAK file would consume the short file name of the PRG and so the modified new PRG file would get a new short name is wrong, because files with different file extensions can all have the same short stem name.

So you actually have to come up with a better example, where that really does happen.

Fair enough.

Just to be clear, I used FoxPro's .BAK as an example of the fact that many programs have ways to save a version that can be used to recover previous versions. FoxPro's method is via .BAK files, MS Office creates hidden files with alternate prefixes, etc.. We can't know for sure what method every editable file will use, and that's just something to be aware of.

If I emailed a file to somebody, and they send it back (with or without changes), then replace the original file with the new one that shares the original name, once again there would be no guarantee the 8.3 name will be the same. If I overwrite it, perhaps. If I delete it and put it back, perhaps, but if other files with the same starting name are added before I put back my file, it would be different.

My main point for the OP is to be aware that 8.3 names are not guaranteed under certain circumstances such as the files being moved to another directory that already has files that share the same 8 character name and suffix, and in that case the OS will definitely create new 8.3 names for any added files that conflict.

 

[Joe Crescenzi]

Here's a quick example of how things have the *potential* to change.

I copied the original file Test.doc multiple times, and gave it a long extension (DOCX) to illustrate some scenarios.

The original test.doc has no short name. When you add a long extension, things get interesting. It needs to generate the short name TEST~1.DOC.

Then I cut a couple of files and pasted them to another location which erased them from the directory and posted them as having ~1 and ~2 in that folder, then copied them back one by one, but not in the original order.

So, now the original folder looks like this, with ~2 now pointing to file 4, and ~4 pointing to file 5.


Starting from this point, I cut test filename3.docx and pasted it into a temp folder:


Now, the file has a ~1, because it's the 1st version in that directory, creating a hole in the original directory where there is no longer a ~3 version, so any file that shares the 1st 8 characters, or a variation of an extension of 4 or more will become the new ~3, and when/if I paste back the old ~3, it will get a higher number.

The main reason for my post was to clarify another member who wrote "If you have "customerzip" and "customername", FoxPro substitutes "custom~1" and "custom~2" to avoid conflicts.", and point out that FoxPro itself is not doing the substitution. It's the operating system that manages those short name, and older versions of FoxPro can only see those 8.3 names and extensions, not create of change them, and that we can't assume they are not subject to change.

 

[Chris Miller]

Yes, it's clear how that happens with moving files around. I still sit on my opinion you can cope with short names if you know nothing like that happens to your file collection for which you maintain references in a DBF. Let alone working on files with long file names when you have no other choice, i.e. have to use files with long names or in paths that (partially) have long names, new extensions with more than 3 letters, etc.

When you rely on being able to USE some.DBF, then you also may rely on the fact that other files are remaining in some application directory and not move change, get overwritten etc. and therefore also can assume to be able to address them by the same short file name, not only by the same long file name. Anything else would make legacy FP applications unusable as soon as file systems support long names and we wouldn't have as many legacy applications still in use.

This whole line of thought went quite a long way away from the actual question, though, where such file names with ~1 come from. That's simply DOS or more generally the OS and filesystem, not a VFP invention, just legacy FP didn't have and need the capability for long names as there were no long names back when that FP versions were new.

And the fact you can still use legacy FP on drives and current Windows versions and that even NTFS has short file names only shows how much MS thinks of backward compatibility. Take aside, that ideally you run a DOS application on a separate DOS installation on a computer and boot to DOS, but there are solutions of DOS integrating into Windows, even Windows 11, which let run a legacy DOS application within a Window that's on a Win11 desktop, too, I think.