Compress XML from web service

[Neil Toulouse]

Hi there!

We have been looking at (and having great success) using web services with out VFP application.

The problem we have hit is that we deal with very large datasets and the XML is taking a fair amount of time to hit the client.

IS there anyway of using any form of compression to speed things up?

TIA
Neil

I like work. It fascinates me. I can sit and look at it for hours...

 

[Olaf Doschke]

Hi Neil,

there is a very old method to have the transfer gzipped with the http protocol. And since Soap Messagesare requested and sent through http, it's quite straight forward to use that feature with web services too. Unfortunately, if you use MSSoap.SoapClient, you cannot interfere the requests sent by it. And if you are not the provider of the webservice, you also have no influence on how the web server will send the soap messages.

Anyway, the idea to use gzip transfer-encoding is laid out here:

http://www-128.ibm.com/developerworks/webservices/library/ws-sqzsoap.html

Bye, Olaf.

 

[Neil Toulouse]

Thanks for the info Olaf!

We had worked out had to set compression on with the web service/server, but had read that you specifically had to tell the proxy to use the compression.

Unfortunately I am using MSSoap.SoapClient at the FoxPro end. Is there anything else I can use to get compression working with VFP?

TIA
Neil

I like work. It fascinates me. I can sit and look at it for hours...

 

[Olaf Doschke]

Hi Neil,

instead of MSSoap.SoapClient you may use West-Winds wwSoap Class:

http://www.west-wind.com/wwsoap.asp

Especially with the wwHTTP class included, you may be able to set the "Accept-Encoding:gzip" header with the soap requests. Take a look at the help chapter "Class wwSoap" -> "Calling Web Services" -> "How to customize HTTP Settings in the SOAP client".

To add the needed encoding header in the request would then be as simple as:

oHTTP = Createobject("wwHTTP")
oHTTP.cExtraHeaders = ;
   oHTTP.cExtraHeaders + ;
   "accept-encoding: gzip" + CRLF

oSOAP = CreateObject("wwSOAP")
oSOAP.oHTTP = oHTTP
...

Ideally that still result in a request response in clear text, if the decoding is handled by wwHTTP, else you'd need to first decode it with gzip yourself.

You might contact Rick Strahl and he'd perhaps be glad to add that feature to wwHTTP and wwSOAP.

Bye, Olaf.

 

[Steve Bowman]

take a look at

http://www.sweetpotatosoftware.com/SPSBlog/PermaLink,guid,9d77301c-460e-44ec-abd5-855d06829b31.aspx

it is a very nice FREE vfp FLL compresion librabry that has a CompressString function and I use it to do exactly what you are wanting to do. I perform my business logic in the web service compressString the xml dataset stream the resulted "compressed" string back to the vfp FAT client uncompress the string then consume the data. this works very well.

My xml dataset was very large and was taking so long to return that users would begin to wonder if the process was hung. The compression pre return reduced my results by over 600% a hugh difference that saved me many issues with users "pushing buttons" when all they needed to do was wait.


Steve Bowman
Independent Technology, Inc.
CA, USA

 

[Olaf Doschke]

Hi steveb7,

this may be sufficient, as it compresses the main load of the soap messages.

Still enabling compression as transfer-encoding of the http protocol would compress the whole package. And you wouldn't have to chage a single line in your application or webservice, as they still get the uncompressed messages.

I'd bet soap will encode the binary data it get's as parameter via base64 and therefore the packets transfered still will be 33% larger than needed, if we could switch on gzip as transfer-encoding.

Bye, Olaf.

 

[Neil Toulouse]

Hi Steveb7!

Thanks for that info! Looks like the way we need to go. Unfortunately we have lost our web services guy at the mo so the project has been put on hold, but I will report back when it takes off again!

Neil

I like work. It fascinates me. I can sit and look at it for hours...