UrlDownloadToFile and OneDrive 2

[JackTheC]

As of today UrlDownloadToFile API for downloading from MS OneDrive does not work anymore. Instead of the requested file a HTML page is downloaded telling to update your browser.

This was known for not updated versions of Windows 8, but as of today this happens too in Up to date Windows 10 and Windows 11, resulting in non working of even crashing applications since the HTML file will get the name of the requested file and you cannot USE an HTML file.

 

[Chris Miller]

In the end, URLDownloadToFile just depends on a URL that represents a file, no more, no less.

But there's more going on before you begin to download from a URL: authentication. Before I dig into it, could you try to add some registry entries for vfp9.exe or yourapplication.exe as described here?

https://dennymichael.net/2015/06/22/web-browser-control-specifying-the-ie-version/

Set the value of the key to 11001 (0x2AF9) to get modenr IE11 behavior.

And then try URLDownloadToFile again.

Other solutions of course are using something else to download files. The safest but surely no the simplest to implenet is going low level and do winsock programming.

Chriss

 

[JackTheC]

VFP applications in Windows 10 and Windows 11 worked just fine yesterday, but as of today Onedrive blocks downloads with URLDownloadToFile, probably because the URLMON.DLL is part of Internet Explorer. I have an alternative but no guarantee that this will be blocked too in the near future.

By the way, downloads from GoogleDrive still work. As long as the requested file is not an EXE. Todays problem is OneDrive specific.

 

[Chris Miller]

Jack, have you tried this?

Chriss

 

[JackTheC]

I'm not talking about the webbrowser control and Windows 10 and 11 already have the latest DLL versions.

 

[Chris Miller]

Have you tried this?

No matter what you think that it's useless. It's taking 10 seconds. Just try it.

And once you got that and it works - for the moment. You can record the winsock commmunication and program that into something futureproof.

Chriss

 

[JackTheC]

Yes, I just tried it but didn't work.

I know of this setting and many years ago I used this for correct rendering webpages that required IE 10 or 11 in the webbrowser control. (IE 10 and 11 is something else then Windows 10 and 11)

 

[JackTheC]

For testing:

declare integer DeleteUrlCacheEntry in WinInet.DLL string
declare LONG URLDownloadToFile IN URLMON.DLL LONG, STRING, STRING, LONG, LONG

* download from MS Onedrive
downthis="[URL unfurl="true"]https://onedrive.live.com/download?resid=XXXXX&authkey=YYYYY"[/URL]  && put in your link
=DeleteUrlCacheEntry(downthis)
=URLDownloadToFile(0,downthis,'c:\temp\myfile.dbf',0,0)   && make sure folder exists

The result is a 79KB file that you can rename to an HTML file and doubleclick to open.
It tells you to switch to MS Edge which I already have of course.

If you manually copy the link in a browser it will work correct. So one can be sure the link is OK.

 

[Chris Miller]

There are two major families of libraries within Windows to use the Internet.
1. WinInet - URLmon is part of that
2. WinHTTP.

Both of them use Winsock on the lowest level.

So one way to use that would be using

https://github.com/calloatti/WINHTTP

or go even deeper and program winsock, as I already said.

Chriss

 

[JackTheC]

Other parts of Wininet like InternetReadFile still work, but of course I don't know how long, since it has the same origin as URLMon.
That was the alternative route I mentioned.

But I guess many programs out there at end users no matter VFP or VB or any other language that try to download from Onedrive will get problems starting from today.

 

[Chris Miller]

I just recommend this as it avoids the problem analysis and instead focuses on finding a solution. We're surely not at the point the whole WinInet branch is not working anymore.

Chriss

 

[Gerrit Broekhuis]

According to MS there is no “new” problem (or they are in weekend modus):

https://support.microsoft.com/en-us...onedrive-36110213-f3f6-490d-8cb7-3833539def0b

Regards, Gerrit

 

[JackTheC]

Gerrit, this is not a bug it's a feature. And in this case that is true. In MS perspective.
Last year they stopped serving IE10 (and older) downloads from OneDrive with URLDownloadToFile.
I thought since IE11 internet tools are an important and much used part of Windows 10 + 11 they would respect that.
And they did for a year or so. But I was wrong. Today they also stopped serving downloads from the latest version of IE11.
Other cloud servers like Google Drive and websites still work.
So I think it is on purpose. Unless some cleaning lady accidently switched a knob on the MS server.... ;-)

 

[Chris Miller]

Jack,

I think the "problem" is that OneDrive is a bit more delicate than just any file on any webserver. You manage access permissions, and even with an authkey=YYYYY in your URL this goes through several steps before you actually get the download. You think about a HTTP GET as making the request and getting the file in response. That's not how HTTP works. Every http request, no matter if GET,POST, HEAD or anything else starts with connecting, then authenticating and then you get a response. And the authentication system is supporting sessions, proxies, and more. If you go through WinInet with Urlmon.dll this is connected to these things with IEs cache etc. If you go through WinHTTP you avoid that. And that's what's worth gold here. Because if something isn't right you get redirected to a login page, that's in the end why URLDownloadToFile gets you HTML instead of the file you expected. It's due to redirection of your request as the authentication step failed.

Edit: I forgot to complete this thought: It's even more obscure as you don't get the HTML of a login page, you get this message to please use the Edge browser, because I guess that login page HTML and JS and other parts it uses doesn't work in this environment.

I notice that you tried to avoid some of the implications of being attached to the IE session by using =DeleteUrlCacheEntry(downthis). I don't think that's sufficient. You can also create links to OneDrive files that work for anybody. Using them for URLDownloadToFile should work. But if you don't want to make a file actually public, then avoiding WinInet and using the family of WinHTTP functions would work. Even more so when working with Winsock directly. Not because that avoids all the authentication, but because it doesn't only avoid the cache, it also avoids the session management of WinInet. And so the authentication process starts from scratch, goes through all steps which finally should accept your request to connect and get the download.

URLDownloadToFile is still fine for many use cases, but not something as complex with the authentication process as a cloud drive is. You can easily grab the logo of tek-tips, for example:

https://www.tek-tips.com/images/header-logo.png

There is practically no authentication process involved in getting that resource as a HTTP response.

Chriss.

 

[Chris Miller]

Here, by the way, is perhaps the simplest form of using WinHttp by using WinHttp.WinHttpRequest.5.1:

Save to UrlDownloadToFile.PRG and let VFP use that by a) not declaring the API function, and b) Set Procedure or Path or both to this PRG.

Lparameters tcNull1, tcURL, tcFilename, tcNull2, tcNull3 && compatible with usage of URLDownLoadToFile(0,URL,Filename,0,0)
*Or rewrite calls and use the simpler parameterization Lparameters tcURL, tcFilename

Local loWinHttp, llSuccess
loWinHttp = CreateObject("WinHttp.WinHttpRequest.5.1")
#Define SYNCHRONOUS .F.
loWinHttp.Open("GET", tcURL, SYNCHRONOUS)
loWinHttp.Send()

#Define OK 200
If loWinHttp.Status=OK and Val(o.GetResponseHeader("content-length"))=Len(loWinHttp.ResponseBody) and Len(loWinHttp.ResponseBody)>0
   StrToFile(loWinHttp.ResponseBody,tcFilename)
   llSuccess = .T.
EndIf

Return llSuccess

Chriss

 

[JackTheC]

For those who come across this page in the future:

Today I found another, and very easy, method for downloading files from the internet including OneDrive.
Just like UrlDownloadToFile you only need one line of code after the declaration :

oScript = CREATEOBJECT("wscript.shell")

cmd='curl -L -s -o "c:\temp\myfile.txt" "[URL unfurl="true"]https://onedrive.live.com/download?resid=XXXXX&authkey=YYYYY"[/URL] '
oScript.run(cmd,0,.t.)

BTW: XXXXX and YYYYY are extracted from the generated embed link for that OneDrive file.

Curl.exe is available within Windows 10 (32 and 64) and Windows 11. So you don't need to install additional software.
For the obsolete Windows 7 and 8 you need to download 2 files from the Curl website: curl.exe and curl-ca-bundle.crt.

 

[Chris Miller]

Well,

you always need two lines and rewrite all your URLDownloadToFile usages. Define a URLDownloadToFile function and your code, which used URLDOnloadToFile, can stay as is, aside from not DECLARing the API function. Did you not get that implication?

Curl, by he way, is available for every vfp user by carlos alloatti, also in his GitHub project.



Chriss

 

[NicolasALLANO]

Hi JackTheC

Your solution is working great! I have embeded it into below function

Private Function DownloadFileCURL(strWebFilename As String, strSaveFileName As String) As Long
Dim oScript, cmd As String
Set oScript = VBA.CreateObject("WScript.Shell")
Dim waitOnReturn As Boolean: waitOnReturn = True
Dim windowStyle As Integer: windowStyle = 1
cmd = "curl -L -s -o """ & strSaveFileName & """ """ & strWebFilename & """"
DownloadFileCURL = oScript.Run(cmd, windowStyle, waitOnReturn)
End Function

And I am testing if DownloadFileCURL(PathSource2, CurrentAddInName) = 0 to assume it went successfully

I have 1 user who is still on Windows 7, and the function did not manage to throw an error. Any recommendation from you?
Thanks

 

[JackTheC]

NicolasALLANO : Like I said, do a manual installation of Curl files for Win7 and Win8:

Curl.exe is available within Windows 10 (32 and 64) and Windows 11. So you don't need to install additional software.
For the obsolete Windows 7 and 8 you need to download 2 files from the Curl website: curl.exe and curl-ca-bundle.crt.

https://curl.se/windows/

(look at the Bin folder of the Zip file)

By the way: I added another parameter in the Curl command: "curl -L -s --ssl-no-revoke -o "
I had one situation (pc or server) that gave error with the original solution, but the above command corrected that. So better use that in your general function too.

 

[Chris Miller]

This looks like VB code, which is not what this forum is about. But if it works, why not?

Chriss