Calling 'send' method of MSXML2.ServerXMLHTTP.6.0" repeatedly

[Rajesh Karunakaran]

Dear all,

I am using MSXML2.ServerXMLHTTP.6.0 for communicating with a API
oHttp = CREATEOBJECT("MSXML2.ServerXMLHTTP.6.0")

I am calling the .open() method, initializes required headers, prepare the input body text etc etc
and then calling the .send() method.

The API endpoint is to create only a single customer record. If I have to create multiple, I need to call it repeatedly.
I assume it doesn't allow to call .send() repeatedly by using a single .open() in the beginning, correct?
I will have to release oHttp after each .send() and start fresh .open(), right?

Rajesh



Rajesh

 

[Mike Lewis]

Hello Rajesh,

I am not at all sure of the answer to your question, but based on my reading of the MSXML2.ServerXMLHTTP documentation, it looks like you call the Open method once, and you can then repeatedly call the Send method. But you need to take account of the Async parameter (the third parameter) to the Open method, to specify whether the sending should be synchronous or asynchronous.

But if anyone tell you anything different from the above, the chances are that they are right and I am wrong.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

Just set up a HTTP service, like IIS or Apache Webserver and experiment with requests to localhost.

You can send multiple times, the question just is how to pass on different customer IDs with each send. The open method defines the URL including URLencoded parameters with ?customerid=x, for example).

So not using multiple open calls you can't ask for multiple customers unless the API gets the ID from the body of the request. If that's the case, you can send multiple bodies. I think you'll just automatically cancel the previous request or not listen to its response anymore. I'd need to experiment myself to see, but one MSXML2.ServerXMLHTTP object can only have one request awaiting a response at the same time.

You can always use multiple MSXML2.ServerXMLHTTP objects to make multiple requests in parallel. Whether that will higher the throughput will depend on how many requests the API will process in parallel. It's likely your requests get queued anyway. If many multiple clients use the API, then even more so you'll likely not get faster or more results that way, but you can always try. In theory, you then at least queue in more often than other clients and get more results per time. But do the same within your application in all your clients and the advantage disappears. You just ramp up the request queue faster for the API, it will only respond with single clients per response to multiple clients and you get the same proportional responses per client up to the maximum the API server can serve.

To even be able to make multiple requests, you have to make them asynchronous, though, and then also have code waiting for the readystate of each MSXML2.ServerXMLHTTP object to signal you got a response, which then takes effort, too.

With an idle server, you can get more out of multiple requests sent in parallel asynchronous, no question. If it's a public API it'll rarely be idle, though.

Chriss

 

[Rajesh Karunakaran]

Mike / Chriss,

I just saw this page
[URL unfurl="true"]https://learn.microsoft.com/en-us/previous-versions/windows/desktop/ms755436(v=vs.85)[/url]

It clearly says,
"You cannot call multiple send methods for a single open method. Instead, for each send method, you must call a corresponding open method"

Rajesh



Rajesh

 

[Chris Miller]

Experiment 1 done.

If you call send() after already having called send() before, you get error "This method cannot be called after the send method has been called".

So you need multiple oHttp objects to send multiple requests in parallel, you need to use them asynchronously. Otherwise, the first object's send() will wait for its response and when you then start the second object's send() it's not in parallel anymore. So parallelism needs asynchronous programming, i.e. you have to have code awaiting the responses for all active parallel requests.




Chriss

 

[Rajesh Karunakaran]

Chriss,

It's a bit different!

I call .open() synchronously (3rd parameter .F.). In fact, we create only a single HTTP object and then with that, for each .send() that we want to use, we have to issue an .open() before that, ie, I see it as a .open()/.send() pair. But, we need only one object.

By the way, if my .send() is working synchronously, I think, my next lines for the next .open() and .send() won't execute until and unless a response is received from my first .send(), isn't it? Otherwise also, I don't really want multiple .send() to be run parallelly.

So, now inside the loop for the table of customer records,
- .open() && with all required parameters
- set all headers
- create the input body Json as per the current record
- .send(body)

Rajesh



Rajesh

 

[Chris Miller]

I see it as a .open()/.send() pair. But, we need only one object.

I know that works, but if you want to send requests in parallel you will need multiple oHTTP objects, there's no way around that, as it's true what you say, a single oHTTP object only maintains one current request and the matching response, so parallelism isn't possible with only one object.

It's all based on Winsock and when you want to go parallel in Wiunsock, you also need multiple Winsock objects.

Chriss

 

[Rajesh Karunakaran]

Chriss,

Yes, of course.
But, for now, I am running it synchronously. So, the present solution is okay.

Rajesh





Rajesh

 

[Chris Miller]

It's not hard to change that. Just to demonstrate:

Local oHttp1 as MSXML2.ServerXMLHTTP.6.0
Local oHttp2 as MSXML2.ServerXMLHTTP.6.0

oHttp1 = CREATEOBJECT("MSXML2.ServerXMLHTTP.6.0") 
oHttp2 = CREATEOBJECT("MSXML2.ServerXMLHTTP.6.0") 

clear
t1 = Seconds()
oHttp1.open('GET','[URL unfurl="true"]https://www.tek-tips.com',.t.)[/URL] && asynchronous
oHttp1.send()
t2 = Seconds()
oHttp2.open('GET','[URL unfurl="true"]https://www.google.com',.t.)[/URL] && asynchronous
oHttp2.send()
t3 = Seconds()
* wait for both requests to respond
Do while oHttp1.readystate<4 or oHttp2.readystate<4
   doevents
EndDo 
t4 = Seconds()
? 'two webpages received after total ',t4-t1
? 'time points:', t1, t2, t3, t4

You'll see the time points t1-t3 will all be close together, as each send is asynchronous, now and receiving means waiting for readystate to get to 4.


Chriss

 

[Chris Miller]

To put this in a simpler working framework, the request sending and the response processing have to be split into two separate methods, and waiting for results could be done by a timer, you can't unfortunately let the oHTTP raise an event in VFP.

I would work this out by starting with a collection of oHTTP objects, every request you do asynchronically will add one more object to it and this object should be told what to call for processing its result aka the responsetext.



Chriss

 

[Chris Miller]

Here's a demo of how to do what I suggest with a collection of requests that is maintained by a class "Requests" you could use in general.

I use nfjson in this, which you can get from

https://github.com/VFPX/nfJson

Copy nfjsonread.prg into the same directory as this prg and CD into it before starting this.
Otherwise you can use the alternative simple StrExtract() code in comments.

For the demo I use the API /api.publicapis.org found in

https://mixedanalytics.com/blog/list-actually-free-open-no-auth-needed-apis/

It requires no API key or other authentication, ideal for such a demo.
You (anybody) might get inspired by the random APIs I fetch here.

Major point of the demo is also to showcase how asanchronous requests can indeed save time. Since the 10 requests run in paralle, the overall processing time is not the total of each single request/response time. In my experience the overall time elapsed is just short longer than the longest single request time, which proves they run in parallel. It also depends on the API whether multiple requests are processed in parallel, if an API queues requests, the time could stack up, instead. But in this case, even though all calls are to the same API, you get 10 requests in almoste the sme time as 1 request.

This has a rate limit, but no worries. The result will then just be no json and the parsing will error, but you'll not be billed for hitting the rate limit, you just won't get results for a while. This varies per API anyway.

#Define XMLHTTP "MSXML2.ServerXMLHTTP.6.0"

Local loRequests
loRequests = Createobject("Requests")

Clear
_Screen.FontName="Courier New"

Local lnStartSeconds
lnStartSeconds = Seconds()

Local lnI, loRequest
For lnI = 1 To 10
   loRequest = loRequests.CreateRequest("GET","[URL unfurl="true"]https://api.publicapis.org/random","StoreAPIs")[/URL]
   * add request header, etc.
   loRequest.Send()
Endfor
? '****************************'
? '* Done sending 10 requests *'
? '****************************'
* The responses will come in and get caught in the Timer() event of loRequests
* Which requires loRequests to stay resident in memory (be in scope) 
* after the code finishes already here

* Therefore a wait loop.
Do while loRequests.oRequests.count>0
   Doevents 
EndDo 
?
? 'Overall time:'+Transform(Seconds()-lnStartSeconds)
* You may instead add loRequests to your goAPP or _screen, and do nothing
* The timer and callback mechanism will work on the responses.

* Callback function (could also use a method of an object)
Function StoreAPIs(lcResponse)
   If !Used("APIs")
      *In the demo store api results into a cursor
      Create Cursor APIs (API M, Description M, Link M, Category M)
      Browse Nowait
   Endif

   Local loNFjson,loAPI
   loNFjson = nfjsonread(lcResponse) && this uses nfjsonread.prg of Marco Plaza's nfjson
   * specifically for api.publicapis.org/random I know this has 1 entry per result
   loAPI = loNFjson.entries[1]
   * get nfjson from [URL unfurl="true"]https://github.com/VFPX/nfJson[/URL]
   * copy nfjsonread.prg into the same directory as this prg and CD into it
   * before starting all this.
   
   * Alternatively, use this simple parsing:
   * loAPI = CreateObject('Empty')
   * AddProperty(loAPI,"API",StrExtract(lcResponse,["API":"],["]))
   * AddProperty(loAPI,"Description",StrExtract(lcResponse,["Description":"],["]))
   * AddProperty(loAPI,"Link",StrExtract(lcResponse,["Link":"],["]))
   * AddProperty(loAPI,"Category",StrExtract(lcResponse,["Category":"],["]))

   If Not Isnull(loAPI)   
      Insert Into APIs Values (loAPI.API, loAPI.Description, loAPI.Link, loAPI.Category)
      ? loAPI.API, loAPI.Link, loAPI.Category
      ? loAPI.Description
   Endif
EndFunc

* General async requests handling
Define Class Requests As Timer
   Interval = 100
   oRequests = .Null.

   Procedure Init()
      This.oRequests = Createobject("Collection")
   Endproc

   Procedure CreateRequest(cMethod, cURL, cCallback)
      Local loRequest
      loRequest = Createobject(XMLHTTP)

      loRequest.Open(cMethod,cURL,.T.)
      AddProperty(loRequest,"cCallback",Evl(cCallback,"This.DefaultProcessing"))
      AddProperty(loRequest,"nInitialseconds",Seconds())
      This.oRequests.Add(loRequest)

      Return loRequest
   Endproc

   Procedure DefaultProcessing(cResponseText)
      *...whatever would suit here
      * usually, you need specific handling for specific API results
      *so you'd make use of the callback parameter of the CreateRequest method
   Endproc

   Procedure Timer()
      Local loRequest
      For lnItem = This.oRequests.Count To 1 Step -1
         loRequest = This.oRequests.Item(lnItem)
         If loRequest.readystate=4
            ?
            ? 'Response after creation of the request in seconds: '+Transform(Seconds()-loRequest.nInitialseconds)
            Local lcCallback
            lcCallback = loRequest.cCallback
            &lcCallback(loRequest.ResponseText)
            This.oRequests.Remove(lnItem)
         Endif
      Endfor
   Endproc
Enddefine

The class Requests is quite well self explanatory with the usage example. Just notice I don't call the send() within the CreateRequest method, as you may need to set headers and add other stuff before you eventually send the request. The Requests class nevertheless already has the request in its collection and will wait for its response within the Timer() event. You need to provide a callback function or method, you see how its eventaually called within the Timer() event with &lcCallback(loRequest.ResponseText), which passes on the ResponseText. To make that a method call instead of a function call it will just requere the callback to be "goApp.method" or simjilar instead of the function name "StoreAPI". Just notice that a local variable name will not be in scope in the Timer() event, so the macro substitution can only call methods of global objects, as it is programmed now.

You can, of course, refine the class and also that callback mechanism and pass in an object and a method name instead to make callbacks by keeping the object reference. It's obvisouly your responsibility that the object still exists, when the response arrives eventually. For function calls a similar responsibilty is that function needs to be findable for FoxPro, i.e. it could be a separate prg in the default directory, findable along the PATH or set by SET PROCEDURE.

Chriss