OLE IDispatch exception code 0 from MSXML3.dll 1

[vfp4ever]

Hello,

until recently I was able to download images from a web site using code like this:

loHttpRequest = CreateObject( "MSXML2.ServerXMLHTTP" )
loHttpRequest.Open( "GET", lcImageAddress, .F. )
loHttpRequest.Send( )
STRTOFILE( loHttpRequest.ResponseBody, lcImageFileName )

Suddenly, while working within the same session, an "OLE IDispatch exception code 0: the connection with the server was terminated abnormally" appeared, and since then I have not been able to make it work again.
The error may change to "... code 0: Access is denied" or "... code 0: Unspecified error" if I try to use a different syntax in CreateObject.

I am using VFP9 SP2 and Firefox on Windows 7 Professional, and step into the same error with a different web browser (specifically as VFP calls the Send method). The images are easily displayed on the browser at their URL address.
At first I thought the msxml3.dll file got corrupted, but using msxml6.dll does not solve the problem.

I suspect it has to do with security certificates, though I have no clue how to proceed.

Dario

 

[Olaf Doschke]

The whole topic of root certificates may not play a role here, just authentication tokens your browser(s) have, but not MSXML2.ServerXMLHTTP. As that's an OLE class dedicated to use of a server that makes HTTP requests to third party services, it isn't involved and embedded into whole area of settings and session cookies a brower has.

As you say you don't get happy with MSXML6, what did you use, what OS do you work on?

Try
   oXMLHttpRequest = CreateObject("MSXML2.XMLHTTP.6.0")
   lcVersion = "6"
Catch
   oXMLHttpRequest = CreateObject("MSXML2.XMLHTTP.3.0")
   lcVersion = "3"
Endtry 
? lcVersion 
oXMLHttpRequest.open("GET","[URL unfurl="true"]https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png",[/URL] .f.)
oXMLHttpRequest.Send()
Do While oXMLHttpRequest.readyState<>4
   DoEvents
Enddo
STRTOFILE(oXMLHttpRequest.ResponseBody, PUTFILE("","google.png","png"))

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Olaf Doschke]

OK, I could confirm my own suspicion.

When I'm logged off from Gmail, for example, requesting the inbox returns an OLE error "Access denied."
[tt]oXMLHttpRequest.Open("GET","

https://mail.google.com/mail/u/0",

.F.)[/tt]

Not logged in to other sites I simply get a redirection and the response body becomes the login page you're redirected to. So it all depends on how the site handles the authentication, it can choose to return an HTTP response header 403 FORBIDDEN, that causes that "Access denied" OLE error.

The MS XML classes are embedded into all the settings, cookies, etc. the Internet Explorer knows, not any other browser, even if you make another browser your standard. I know, because I did. All the MS classes only have access to the MS realm of internet-specific things, not where and how Chrome or Firefox knows your sites long term session cookies that automatically log you in. So no matter whether you use msxml3 or 6, the ServerXMLHTtp class or the "normal" XMLHTTP class, a site will only let you get a resource when you're logged in, some sites cause an error, others a redirection.

It's nothing wrong about the DLLs, nor is your certificate store corrupt.

I don't know the detail of your URLs. When you request images, that are public, that shouldn't be an issue- The simplest thing to try is visiting the site with IE and when necessary, log in. Then try to request what you want to access with VFP code.

And last not least, be very strict with your URL. It does make a difference if you request https:// vs

http://,

some sites may respond with forbidden status to any http:// requests, most of the internet today runs on

https://.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[vfp4ever]

Many thanks for your reply, Olaf.

I am on Windows 7, and the image URLs are from an online stamp catalogue (e.g.

https://i.colnect.net/o/153/157/Int-Year-for-the-Child.jpg).

Logging in to the website doesn't make any difference as for image downloading, and I can still display any image with http:// either.

As I said earlier, the problem arised at once while I was using my VFP application, so I doubt there was any change in security issues at that very moment. Nor was I changing my code or anything that could impact such behaviour.

 

[mplaza]

If you use Windows Server 2012, Windows 7 Service Pack 1 (SP1), and Windows Server 2008 R2 SP1. you should update to latest service pack to connect to sites using TLS 1.1 or TLS 1.2.

You can also add TLS 1.1 & TLS 1.2 support using this quick fix ( must be using windows 7 sp1 at least ):

https://support.microsoft.com/en-us...and-tls-1-2-as-default-secure-protocols-in-wi

Marco Plaza
@nfoxProject

https://www.github.com/nftools

 

[Olaf Doschke]

I doubt there was any change in security issues at that very moment.

A sudden behaviour change will not be caused randomly, but by, well, a sudden change of anything. Marco is likely on the right track that any upgrade in the site about the SSL had an impact, or a Windows update. Is Win7 out of support phase by now?

Yes, it is:

https://www.laptopmag.com/articles/windows-7-end-of-life-guide

That's how such base libraries involved in establishing SSL tunnels don't get updated anymore. But sites update theirs.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Olaf Doschke]

And about trying http;//, it turns out i.colnect.net is redirecting such requests to

https://,

so you can't choose "I waive for a secure connection to you, hand me over that picture without establishing a secure tunnel."

It also seems to me, then, the whole mechanism of establishing an SSL connection is simpler, more bound to one choice of TLS, than what the IE itself supports. And Firefox is known to even not use the cryptographic libraries of Windows.

The way SSL connections are established starts with a handshake procedure that negotiates what TLS version to use.

Here's a bug report mentioning update MS14-066:

https://serverfault.com/questions/741706/bug-with-tls-1-2-in-internet-explorer-11-windows-7

The simplest way out would be updating to Win10, though.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[vfp4ever]

If I open my web browser's (Firefox) configuration page, under security.tls.version.max, I can see that it's already set to 4, which should mean that TLS 1.1 and 1.2 are installed and working. Yet I have installed the Quick Fix Marco has suggested, and installed all the recommended Windows updates (apart from Silverlight and .NET 4.8) that I left behind. Unfortunately the error is still showing.

 

[Olaf Doschke]

You don't get the latest Updates of Microsofts cryptographic libraries, as said, Win7 is out of support.

And fine, Firefox has it's own libraries, Firefox doesn't use Windows foundations and IE doesn't use Firefoxes foundations.

The easiest way to fix your problem will be a Windows upgrade to Windows 10.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[vfp4ever]

Please forgive me if I got it wrong. Olaf, are you saying that the MSXML3.dll (as well as MSXML6.dll) are giving me errors because of some obscure library that Windows 7 has not updated since last month (and is not going to update until the end of scheduled support later in January)? And I should update to Windows 10 to get it fixed back? I assume you have tried the above code and it works fine on your system.

Well, my knowledge of networking and internet services is very limited, yet I thought that those instructions that let me download an image file were very basic. Is there any other way to do it under Windows 7?

 

[Olaf Doschke]

It's the only conclusion I have. But indeed the end of extended support is still not yet, it's January 2020.

So, of course, the possibility is, it only is a thing on your Win7 based computer, then others may help you find out, but I'll not go back to Win7 to see what's wrong.

You could use tools like Fiddler to maybe find out more about where requests to the pictures fail, you may step on your own foot outside of your system in a hardware firewall of your router, for example. But it's quite clear you have a problem establishing the connection to the server.

I'm pretty sure though when you make the move to Win 10 - and it's pretty close to the deadline 2020 - you revive your system in that aspect and solve two concerns at once. Of course, you would check first, whether your system can run Win 10, I couldn't reuse a TPM module, but that was a minor issue.

Bye, Olaf.


Olaf Doschke Software Engineering

https://www.doschke.name

 

[vfp4ever]

I have been desperately looking for a solution, but with no success.

What I don't understand is why the same few lines of code work for this image (

https://www.nginx.com/wp-content/uploads/2018/09/MemberCentral-Terry-235x235-min.jpg),

but fail for this one (

https://i.colnect.net/t/5532/331/Timbre-movil.jpg).

As a matter of fact, a "View Image Info/Security" reveals that under "Connection Encrypted" I can see the very same data, specifically TLS_AES_128_GCML_SHA256, 128 bit keys, TLS 1.3.

I contacted the website master and he told me he had redeployed his domain with NGINX, and seemingly it appears that MSXML hasn't worked since then. But I have no problem downloading the face of NGINX's main developer!

Olaf, I know you will not install Windows 7 to see if it works or not, obviously I would never expect anyone to install an OS to solve such an issue. As, I believe, it would not make sense to purchase a brand new machine with Windows 10 only to see whether it works or not, would it? There must be something else that I missed, maybe I should invoke the HttpRequest.SetRequestHeader( ) method with specific instructions to have Send( ) respond more politely?

 

[Olaf Doschke]

First, I didn'T suggest buying new hardware, just Win10, and that starts at affordable prices. It'd be a topic anyway, unless you don't care for Win7 going out of support.

I assume you do the "View Image Info/Security" in your Firefox browser? I already shed a light on why all this works in Firefox, it doesn't make use of OS cryptographic libraries. What it does prove is that both servers support TLS 1.3 One of them might step down to TLS 1.2 or 1.1 if the client side says he can't. I'm not in there deeply, but this is one possibility. The libraries at work from Win7 and the current situation in your Windows state might mean a server not stepping down to a less strong encryption in the negotiation phase between client and server how to establish the SSL tunnel for a HTTPS connection.

No SetRequestHeader won't help, the stage at which your header arrives at the server side is way after making the HTTP connection, because that header also already will be transported through the SSL tunnel established in the process beforehand. This is not a rejection because of credentials missing, but because your Win7 can't negotiate on using some version of TLS for making the HTTPS connection. And I doubt even an experienced web master will be able to help you from his work and configuration of a site, establishing the https connection and ssl tunnel. The level on which this is configured is the webserver, not php.ini or any login script in perl,php,python or node.js or whatever. In apache for example, this controls what is allowed:

https://tecadmin.net/enable-tls-in-modssl-and-apache/

That article describes how to disable all older possibilities and enforce a minimum of TLS 1.2, for example.

Your false assumption is that Firefox browsing proves the MSXMLHTTP class must work with the same URL. Firefox is even able to use sites enforcing TLS 1.3, but you don't know by that, whether MSXML on your Win7 also supports taht same security strength.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[atlopes]

Vfp4ever

It seems a case where W7 can't cope with the TLS settings of the second server - and it was in the server that most likely any changes may have occurred. W10 is ok with those settings, W7 is not.

I just tested using the Chilkat HTTP library in a W7 machine and it is able to download the second image, so you have that possibility that I strongly recommend (and that you can try for yourself without cost).

 

[vfp4ever]

OK, now I am leaving MSXML behind for good and trying with curl command-line utility, hoping that I can successfully call it with loScriptShell.Run( "...", SW_HIDE, F_WAIT ) and have my silly image downloaded:

curl [URL unfurl="true"]https://i.colnect.net/t/2050/870/Tank-farm.jpg[/URL] -v -tlsv1.2

But, after the ritual hellos and handshakes, I cannot see further than a "(56) OpenSSL SSL_read: No error", which from curl's manual translates into a "failure in receiving network data". If I use -tlsv1.3 the server responds more rudely without any handshake, giving "(35) error:14094410:SSL routines:ssl13_read_bytes:sslv3 alert handshake failure".

My assumption is that curl does not rely on obsolete Windows 7 services, just as Chilkat libraries (that do work on Windows 7, according to atlopes). So, what is wrong this time?

 

[atlopes]

Vfp4ever,

The server does not like to be requested by a CURL. You have to impersonate yourself as a browser agent.

curl [URL unfurl="true"]https://i.colnect.net/t/2050/870/Tank-farm.jpg[/URL] -A "Mozilla/5.0" -v --output d:\temp\xxx.jpg

 

[Olaf Doschke]

If that's the only real reason, the agent name, as atlopes says, MSXML will use an agent name, so that can't be the only reason.

What you see points out the problem is as I suspected about creating the SSL tunnel, the handshake provess is part of that.

And I doubt that curl like Firefax is using own libraries, that's very specific to Mozilla. Curl might make us eof openssl libraries. But since you have the same problem with it I assume it actually binds to the OS layer. Curl is available for multiple platforms and AFAIK compiled with GCC, which in itself is available for mulitple platforms and will bind to OS specific APIs, usually.

So there you have that confirmed.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Olaf Doschke]

To be precise: When you use MSXML2.ServerXMLHTTP What arrives serverside as Agent is:
[tt]HTTP_USER_AGENT=Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)[/tt]

When using Microsoft.XMLHTTTP the Agent string will be
[tt]HTTP_USER_AGENT=Mozilla/5.0 (Windows NT 6.2; WOW64; Trident/7.0; rv:11.0) like Gecko[/tt]

The core point is it's not empty.
And don't confuse this as being Firefox based, just because Mozilla is mentioned. Compare that to what is given here:

http://www.useragentstring.com/pages/useragentstring.php?name=Internet+Explorer

The COM objects all work in the same realm as Windows Internet Explorer.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[atlopes]

Vfp4ever and Olaf

To clarify: it seems that the server in i.colnect.net is not accepting requests by a CURL application. CURL sends an agent signature, it's only that the server i.colnect.net is not willing to respond to requests that use it. Hence, the need for CURL based applications to use a different signature.

So, Vfp4eever, summarizing:

For W7, use Chilkat or CURL (in this case, provided that non-CURL agent signature is used).

For W10, MSXML2 is ok.

 

[vfp4ever]

Thank you, atlopes!!! Now that does work! With just one single line you've made my day (or week, or even month).

I had no idea one has to disguise as a browser to download a plain image file, though I had the feeling something else would have worked it out somehow. Great answer indeed. And great tool curl is.

Olaf, it is always sheer pleasure for me to read your detailed explanations, but you seemed too pessimistic this time. Thank you anyway for your kind help.

Regards to all,
Dario