Accessing FAT32 from within VFP

[vfp4ever]

Hello,

I am developing an application to organize my music collection of .mp3 files, play them, read and write their ID3v2 tags, create Playlists and so on.

Now I am facing the known problem of keeping a USB flash drive (FAT32) with folders and tracks in the preferred order (most likely: artist folders in alphabetical order, song files in order of year-album-track number), so that I can easily look up an artist and play his entire album without driving my car into the ditch. I did find some simple utilities (along with source code in C or Phyton) that do that, but they only sort alphabetically by song title (they do not bother with tags). I was successful in moving all tracks to my hard-disk and then back (in a certain order) but, apart for the lengthy operation, I still cannot get folders sorted properly even after renaming them, or creating/deleting them (in order). I believe the definite solution lies in sorting the FAT.

Before venturing into low-level C code that I have almost forgotten, and try to translate it into VFP structures and lots of FREADs, I was wondering if anyone has ever tried to read (...and write back!) a File Allocation Table before. Please forgive me if it sounds too much of a challenge (I am only giving it a try...).

Regards,
Dario

 

[Olaf Doschke]

I wouldn't do that, it's a too deep dive into file system specifics. And then only for FAT32.

The normal way to sort data as you like is storing it as records you can index in serveral sort orders. In this case of course that data to sort by would not be blobs of the files, just file names, but also any tag info you want to sort by. That this is only possible in your player application should play a minor role, or are you really after sorting files so any application or car mp3 player would list them this way?

Then I fear you have to dig into FAT tocs, yes. This is better asked in some hardware/driver developer forum. I know this exists, but haven't done what you hint on having done long time past.

One way you might try is writing shortcuts to the mp3s in one sort order folder, so you don't read and write all full mp3 files. But that would require your hardware to also play a list of shortcuts to mp3s.
Is there any chance that player also reads and play playlists? Then creating playlists is an obvious solution not needing any reorganisation of files. That merely compares to the DBF you'd use in a VFP application but ends up as a text file management task only. I know many players do suppport m3u lists, for example.

Bye, Olaf.

 

[vfp4ever]

Hi Olaf,

unfortunately car MP3 players are not that smart, at least not until a few years ago, but not that stupid enough to be fooled by a folder shortcut appended with extension .mp3 either. Actually mine can even see hidden directories... but ignores playlists (.m3u files), and its screen text is so limited that even saving a single character is worth it. If I could manage to selectively sort folders only with those simple utilities, leaving out the files contained in them, then I would not think to bother with the file system.

I know this problem is common to most portable (cheap?) MP3 players, including those of middle-priced car brands. If I could work it out, I am sure it would be of help to many, though it is now intended for my personal use. I will follow your suggestion to search the forum and look at other languages.

 

[Mike Lewis]

If by "sorting the FAT" you mean actually re-arringing the blocks of data within the file system, I would seriously doubt that you need to do that. I don't have a lot of experience with dedicated MP3 players, but it does seem to me to be a drastic solution.

What you need to do is to figure out what order the player plays the files in. You say it doesn't support a play list. So does it play the files in alphabetical order of file name? Or of track title? Or in the order in which you placed the files on the device? Or what? After that, the next step is to get the files into the relevant order.

Admittedly it's probably more complicated than that, given that you want to play by, for example, title within artist. I'm sorry I can't give a definite solution to that, but I do feel sure that physically sorting the data within the FAT is not the way to go.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Olaf Doschke]

From what he said the player can play files in the order they were written and that is the current cumbersome and slöow solution.

I rather think he only wants to address the directory infos on the low leve of the file allocation table itself, which just holds file names and some pointer to a first file block. If that is a simple list in a certain order you may switch any two entries to sort files differently, without moving around file blocks themselves.

It is a file operation you can't even do with VFPs low level file functions FOPEN() to FCLOSE(), this is even lower level and unless you don't find some API function calls supporting FAT access I don't know if that's possible with VFP at all.

Bye, Olaf.

 

[Mike Lewis]

From what he said the player can play files in the order they were written and that is the current cumbersome and slöow solution.

OK, I missed that.

So, Dario, are you wanting to establish your playing order at the point where you copy the files to the device? Or do you want to be able to arbitrarily change the order - or pick specific tracks to play - once all the files have been copied?

If the former, I guess it's a question of deleting all the existing files from the device, deciding the order of the ones you want to play (presumably by reference to their ID3v2 tags), and then copying them back to the device individually in the specified order. You can do all that in VFP of course.

If the latter ... I can't see any way of doing that.

I suppose the whole thing depends on how badly you want this - and how much you are willing to pay for it. I've no doubt that some of the newer phones and tablets will let you set up play lists and respond to voice commands where you tell it which tracks you want. If you were willing to buy one of those, you could perhaps connect it to your car's audio system via its Aux In port.

On the other hand, I might be talking rubbish. I've absolutely no experience of listening to music in a car. It's not something I've ever wanted to do.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[GriffMG]

Could the OP not rename the files to start with a numeric code, this would probably then sort nicely?

001 The Police - Every step you take.mp3
002 Anastasia - Not that Kind.mp3
003 The Kinks - You Really Got Me.mp3

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

I'm trying to cut down on the use of shrieks (exclamation marks), I'm told they are !good for you. ​

 

[Olaf Doschke]

its screen text is so limited that even saving a single character is worth it

For that reason a prefix will waste some valuable characters.

I was successful in moving all tracks to my hard-disk and then back (in a certain order)

As that works, the file play order by default is the order in which the files were put in the FAT, not alphabetical order.

File time might also be the attribute sorting the play order in the chronological write order, that gives me another idea: What you could try, vfp4ever, is setting file time via the FoxTouch function of foxtools, see

http://www.alvechurchdata.co.uk/hints-and-tips/foxtools-fll.html

Last not least, while we talk of time: All the time you invest in this also is worth money, is there no other player available fitting into your car? Even if you think you can earn some money with this tool - how much do you think people would pay, if they only invest in a cheap hardware player? I assume m3u support should not be hard to get and already mid class players would have that, if not all current players.

Bye, Olaf.

 

[vfp4ever]

Thanks all of you for your feedback.

Perhaps I should have told you earlier that I already tried all possible means (including changing file and folder times) to get these MP3 players sort music files in a meaningful order, until I found a detailed explanation last week by somebody who developed a "FAT32 Sorter".

Yes, Mike, the majority of these players' firmware (not only my car's) take the natural placement of entries in the FAT as sort order, so there is absolutely no way of controlling it changing file attributes or ID3v2 tags. The only thing I can easily do in VFP code is copy the whole drive (e.g. 32 GB) back to my computer, one folder (artist) at a time, delete the original folder, sort its files the way I want (detecting ID3v2 tags) and finally copy them back, one file at a time, into the newly created folder with the same name. If I do not delete-and-create the folder, the order will be lost. But the problem of having the folders unsorted remains... So I can play the albums "Fragile" and "Close to the Edge" as I would normally do from their CDs at home (in order of year-album-track number), but I will still find the artist "Yes" randomly placed between "Kinks (The)", "Atomic Rooster" and fifty more names.

Olaf, if you say I cannot use low-level VFP functions (FOPEN, FREAD, ...) for this purpose, then I believe I should rather use an external C routine. I have not seen any special API function other than FindFirstFile and FindNextFile, so I had better look at the FAT32 Sorter source code first.

Dario

 

[Olaf Doschke]

you say I cannot use low-level VFP functions (FOPEN, FREAD, ...) for this purpose

Yes, for the simple reasons these file functions work on files themselves only, not on the FAT: VFP functions of that family are (in alphabetical order) FCHSIZE,FCLOSE,FCREATE,FEOF,FFLUSH,FGETS,FPUTS,FREAD,FSEEK, and FWRITE. None of them about the FAT, they all work on FAT,FAT32 and NTFS, too. The file system handling really is delegated to Windows even from the VFP runtime.

And yes, seems like you need a C DLL you may use with VFPs DECLARE DLL to have necessary functions or an EXE with command line interface.

Bye, Olaf

 

[Mike Lewis]

For what it's worth, I found this:

FAT File System API
Low-leve FAT file system access for Windows Mobile devices


But it's not clear whether it is a commercial product or just an example of someone's work. You've no doubt found other possibilities yourself, Dario.

I would only add that if there is a Windows API or a third-party DLL to do the job, then it should also be do-able in VFP.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[vfp4ever]

Yes, Mike, I believe it can be done in VFP using a few Windows API functions, as I am now experimenting.

I can successfully access the external device (USB flash drive in this case):

lnHandle = CreateFile( "\\.\F:", GENERIC_READ + GENERIC_WRITE, ;
FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0 )

I can dismount and lock it:

DeviceIoControl( lnHandle, FSCTL_DISMOUNT_VOLUME, 0, 0, 0, 0, @lnReturned, 0 )
DeviceIoControl( lnHandle, FSCTL_LOCK_VOLUME, 0, 0, 0, 0, @lnReturned, 0 )

then the original C++ code gets the drive "geometry" (bytes per sector, sectors per track, etc...):

DeviceIoControl( lnHandle, IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, 0, 0, @lcDiskGeometry, ;
LEN( lcDiskGeometry ), @lnReturned, 0 )

but at this point I have not been able to supply the correct structure (with the old DRIVE_GEOMETRY it works). The problem is that the new DRIVE_GEOMETRY_EX structure, which incorporates the old one, has a variable-sized length. I will have a closer look at how complex structures work in VFP.

 

[Olaf Doschke]

In the end you always prepare a C struct via "poking" bytes into a string you pass in or get out by reference and then "peek" single results out. Well, you know that, you do it with @lcDiskGeometry.
The well known news2news site covers usages of API function also with structs, see

http://www.news2news.com/vfp/?group=-1&nothing=0

for an entry point in the whole overview.

They typically use their own conversion functions, their samples are older than BINTOC() and CTOBIN() where implemented in VFP.

DeviceIOControl:

http://www.news2news.com/vfp/?group=-1&function=866

Obtaining physical parameters for a drive: sectors, clusters, cylinders...:

http://www.news2news.com/vfp/?example=101&function=866

Let us know, if that works.

Bye, Olaf.

 

[Nigel Gomm]

did you try playing with the extended file properties - index 19 (the track number)?

https://technet.microsoft.com/en-ca/library/ee176615.aspx

i remember having a similar issue with playlists exported to an android device.

hth

nigel

 

[vfp4ever]

Nigel,
thanks for the tip, I tried with those extended properties already, even if I could not see any usefulness. After a few more searches, I can say that it is unfortunate that the GetDetailsOf method of the Shell.Application does not have a SetDetailsOf counterpart to set those properties. I seriously doubt any hardware manufacturer would ever rely on them.

Olaf,
I have been slowly making some progress in using those API file functions, somehow getting slightly optimistic.

I was testing some code taken from Calvin Hsia's blog (

https://blogs.msdn.microsoft.com/calvin_hsia/2005/03/18/calling-the-windows-apis-for-large-files/)

when I ran into a Syntax error that I cannot explain myself. I was trying to split a 64-bit LARGE INTEGER into the component members DWORD-LONG:

LOCAL lnLargeInteger, lnLowPart, lnHighPart AS Long
lnLargeInteger = 2 ^ 32 + 1
lnHighPart     = INT( lnLargeInteger / 2 ^ 32 )
lnLowPart      = lnLargeInteger – lnHighPart * 2 ^ 32       && I get a "Syntax error" here

Any explanation?

Dario

 

[Olaf Doschke]

No idea, but you would use modulo (% operator) anyway:

LOCAL lnLargeInteger, lnLowPart, lnHighPart as Long
lnLargeInteger = 2^32 + 1
lnHighPart     = INT( lnLargeInteger / (2^32) )
lnLowPart=lnLargeInteger%(2^32)
? lnHighPart, lnLowPart

But this has it's limits. In VFP every numeric variable in memory is float, integers are only field types and bigints or Longs are not a supported native type. LOCAL var as type is not a type declaration, it's only a syntactic sugar to get intellisense, but no intellisense exists for simple types as Long.

What you would do in VFP to handle 64bit numbers is use binary type:

LOCAL lnLargeBinary
lnLargeBinary = 0h0000000100000001
lnHighPart = EVALUATE('0x'+LEFT(TRANSFORM(lnLargeBinary),8))
lnLowPart = EVALUATE('0x'+Right(TRANSFORM(lnLargeBinary),8))
? lnLargebinary, ''+lnLargeBinary, '0x'+Transform(lnLargeBinary), EVALUATE('0x'+TRANSFORM(lnLargeBinary)), lnHighPart, lnLowPart

When you pass in and out structs via strings bytes, don't try to extract longs nor to put them in these struct strings, you can extract any portion of bytes and use CREATEBINARY(bytes) to turn them into a binary type, which you might evaluate via Evaluate('0x'+Transform(binary)), the other way around (numeric data to bytes) is more difficult, but you can convert partial numerical values to the corresponding string bytes via BINTOC(), which behaviour is best illustrated as follows:

? CreateBinary(BINTOC(0x04030201))
? CreateBinary(BINTOC(0x04030201,"4"))
? CreateBinary(BINTOC(0x04030201,"4S"))
? CreateBinary(BINTOC(0x04030201,"4R"))
? CreateBinary(BINTOC(0x04030201,"4RS"))

So if you want the normal big endian (most significant byte first) the correct BINTOC flags are "4S". The default conversion (without flags parameter) acts as the call with "4" flag and is meant to be usable as index value for machine collation, so negative numbers sort before positive in ascending order, therefore by default you get the inverted sign bit, also see CreateBinary(BINTOC(-1)) and CreateBinary(BINTOC(1)). It's not necessary to index an int field, but to have a compound index on BINTOC(ifield1)+BINTOC(ifield2), as VFP doesn't have real compound indexes on a list of fields but you can index on any expression, which offers things comparing to MSSQL indexes on computed fields without the computed fields themselves. Besides that function would better be called NTOC() or NumToChar(), but that aside.

You're allowed to put "8" in the flags parameter, too, but that doesn't act on longs (64bit longs sometimes also referred to as long longs or bigints), see for yourself:

? CreateBinary(BINTOC(CAST(0x04030201 as N(19)),"8S"))
? CreateBinary(BINTOC(CAST(0x04030201 as float),"8S"))
? CreateBinary(BINTOC(CAST(0x04030201 as double),"8S"))
? CreateBinary(BINTOC(CAST(0x04030201 as currency),"8S"))

In the end you can only easily inspect the value of more than 4 bytes by looking at their hex representation using the binary type - which you create from the string or string snippet via CREATEBINARY(cData), as said. Since VFP supports binary, varbinary and binary memos that means the length for that is quite unlimited for memory variables, eg lcBin = CREATEBINARY(SPACE(16000000)) works. In the end it's the same bytes as a VFP string variable holds, just displayed in hex with 0h prefix.

Bye, Olaf.

 

[vfp4ever]

Thanks a lot, Olaf

I will definitely study this case and do some experiments with huge (binary) numbers. Will keep you informed of what USB flash drives have to say...

Dario