VFP and SOAP for webservices - from a SOAP newbie 3

[KarenLloyd]

Hi there Gurus

Can I seek your advice again please?

I have been asked to interface between a VFP app and an established web service using HTTP and SOAP. While I have done some previous work with HTTP requests, I have not to this point done any work with SOAP.

I have info relating to the structure I need for this particular web service, but I have so many questions - I don't know where to start to build this into my system.

My very basic understanding is that I would create an object in a similar way as with HTTP, and send a file string to the web service, accept a response and process this to retrieve the data.

I need to get onto this as quickly as possible, and have spent a couple of afternoons searching the web, reading up on options and trying to set in my mind a place to start, but have some concerns about getting the right tools. And being behind the times, I don't want to start at the wrong place.

My first issue is that my application is written in VFP6. I have so far avoided upgrading as VFP6 did the job it was needed to do, but it is now considered "ancient" according to some. The later versions of FoxPro have so much more webservice functionality, so to start off - my questions are...

Q1) Will VFP6 be able to work with SOAP? (If not, then I'll have to upgrade to VFP9 - why mess with 7 or 8, right?)

Q2) If I have to upgrade - can I expect the conversion from 6 to 9 to be fairly straightforward? This application is fairly simply written - i.e. no transaction processing, some SQL selects, major use of grids, but fairly basic / old style programming - not an awful lot of OOP or ActiveX (if that makes sense).

Q3) What else do I need to get started? I read that the MS SOAP Toolkit is no longer supported - or am I wrong..?

Q4) What about Windows versions in general? While I have XP / Vista and Win7 at my disposal - isn't XP "expiring"? What should I be testing on?


It's a dilemma so far. But rather than feel negative, I would like to start from the right place and be able to plan ahead.

I have been given access to the webservice for testing and I want to start as soon as I can.

Please can you give me any opinions as to what I should look to next, based on your wealth of experience and my lack of it?

Thank you for your time.

Kind regards

Karen

 

[Mike Lewis]

Hi Karen,

I'll take a shot at your questions. But I expect other people here will give you better answers than I can, so don't take this as the final word.

Q1) Will VFP6 be able to work with SOAP? (If not, then I'll have to upgrade to VFP9 - why mess with 7 or 8, right?)

As far as I know, there's noting in 6 that will prevent it working with SOAP. But it's very much easier in 9, because you will be able to use Intellisense to generate much of the code for you. And if you are going to upgrade, it makes perfect sense to go to 9 rather than 7 or 8 (which in any case you might not even be able to buy now).

Q2) If I have to upgrade - can I expect the conversion from 6 to 9 to be fairly straightforward? This application is fairly simply written - i.e. no transaction processing, some SQL selects, major use of grids, but fairly basic / old style programming - not an awful lot of OOP or ActiveX (if that makes sense).

The main thing to remember is to issue [tt]SET ENGINEBEHVIOR 70[/tt] at the start of your program. That way, your SQL will work the same as before. But there might also be a few other minor incompatibilities, so you will need to re-test the entire application carefully. But it's not likely to be a big problem.

Q3) What else do I need to get started? I read that the MS SOAP Toolkit is no longer supported - or am I wrong..?

I don't know if it is still supported, but it certainly still works. I have used it a lot recently.

Q4) What about Windows versions in general? While I have XP / Vista and Win7 at my disposal - isn't XP "expiring"? What should I be testing on?

No, XP isn't not "expiring". Rather, Microsoft is withdrawing support in April. XP will continue to work after that, but you will no longer get updates or security fixes. It's up to you to decide whether using XP on that basis is better than going to the considerable effort (and cost) of upgrading. It's an important decision, but it's got nothing to do with VFP or SOAP.

I hope this will be of some help. If and when you install VFP 9.0, let us know, and I (or someone) can talk you through the steps of using Intellisense to generate your web service code.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[JRB-Bldr]

I too can only offer some help.

I have built a number of SOAP interfaces with 'outside' entities and a number of them were 'wrestling matches' requiring a LOT of external entity technical support.

Unfortunately all of my code references are at my home office which I cannot get to until later, but until then here are a few references to look over.

Rick Strahl's: Building and using a SOAP Web Service with West Wind Web Connection

http://www.codemag.com/article/0101051

Another Rick Strahl article: Visual FoxPro Web Services Revisited

http://www.codemag.com/article/0703062

MS SOAP Toolkit is no longer supported
As Mike indicated above, it still works and I too was using it for my client projects.

You might also want to look at:
How to Test a Web Service – Its all about the WSDL

http://www.joecolantonio.com/2011/04/13/how-to-test-a-web-serviceiwsdl/

How to Test a Web Service – Response and Requests

http://www.joecolantonio.com/2011/04/21/part-3-–-how-to-test-a-web-service-–-response-and-requests/

Lastly since I use Chilkat utilities I got some initial direction from these sites as well:
HTTP Visual Foxpro Examples

http://www.example-code.com/foxpro/http_soap.asp

http://www.example-code.com/foxpro/soap_web_service.asp

If you haven't gotten all you need by then, I'll try to get back later today with more specific details on the projects I have done.

Good Luck,
JRB-Bldr

 

[KarenLloyd]

Hi Mike

Thank you so much for giving an answer to each of my questions.

I am concerned at this stage about the overheads involved in having to re-test an extensive suite of programs after converting from VFP6 to VFP9. I am less worried about coding the elements myself, as that is the way I work most of the time and I find it gives me greater control over what I want to achieve.

I will bear your responses in mind as I consider my options over the rest of the week.

At least if I can download the latest utilities then I will have something to start with.

I really appreciate your help once again.

Best wishes

Karen

 

[KarenLloyd]

Hi JRB-Bldr

Thank you very much for your response.

I have had a brief read of some of the items from Rick Strahl before and looked at Chilkat earlier this afternoon.

I have been given the means to test the web services by the third party company, which is great, I just want to make sure to get the right tools in place for accessing the web service.

Thank you for confirming that SOAP Toolkit is still functional.

I will read through the links you have included for me.

I am grateful for your input, thank you.

Best wishes

Karen

 

[Neil Toulouse]

Hi Karen

Just to add to the advice above, last year I used the SOAP Toolkit in our app to interface with an external WebService and it worked fine (VFP 8). Microsoft gave up on the SOAP Toolkit some time ago in favour of the .NET implementation. I then tested it on all versions of windows (desktop and Server) and again it worked fine.

However...

We host the app for a number of customers on some complex setup, and the SOAP Toolkit would not install on that setup (no idea why and no one could tell me!). The teccies just beat me up over using something that Microsoft no longer support (I kept quiet about VFP... ). Therefore I was 'forced' to create a .NET dll instead. That was quite painful in comparison, but I gained a lot of info from another excellent Rick Strahl article [URL unfurl="true"]http://www.codemag.com/Article/0703062[/url] and got there in the end!

...and guess what? They can't get that dll installed either... So nothing's wrong with the toolkit.

Also watch out if your web proxy needs authentication, as that may cause you grief during development!

Neil

I like work. It fascinates me. I can sit and look at it for hours...

 

[KarenLloyd]

Hi Neil

Thanks very much for your input and the extra link. Another thing I will run up against is some version snobbery, and comments such as "what year is that from?"

I don't want to go into .NET if I don't have to. But this is becoming urgent and I don't feel even close to getting started yet.

I will be testing from my office before I even think about testing on the client's system. They are using Windows Server 2003 for Small Business Server, but have their own IT company to sort out the web proxy side, so I guess that's going to help some.

Still more than a little overwhelmed, but once I make the first "contact" I'm sure I'll feel better about the whole thing. Here's hoping anyway.

Best wishes

Karen

 

[Mike Lewis]

Karen,

We were talking earlier about whether you would need to upgrade to VFP 9.0 - and the need to re-test the application if you do. I've had a couple of further thoughts on that subject.

I said that there was nothing in VFP 6.0 that would prevent SOAP from working, but it's very much easier in 9.0. That's true. But if you are going to be working with web services, the chances are that you will also need to work with XML. VFP 6.0 has no real support for XML (apart from writing your own code to parse and translate it), whereas 9.0 does have reasonable XML support built in. So that would be another advantage of upgrading.

If you want to avoid having to re-test the entire application, can you isolate the SOAP / XML function into a separate module? If so, consider creating a separate executable for it. Keep the main application in 6.0, and just use 9.0 for the new stuff. It might not be the ideal solution, but it might be worth considering.

Finally, I've been looking through the list of incompatibilities between 9.0 and 6.0. It's a long list, and unfortunately it's not contained in a single place. You would have to read the What's New sections in the Help for the three separate versions, (7, 8 and 9). On the other hand, the vast majority of the changes are very minor - not to say trivial. Or, they are concerned with tightening up syntax that was previously week or ambiguous.

To give you two examples. First, [tt]_SCREEN.Toolips[/tt] now defaults to .T. rather than .F. I can't imagine that being a problem, because if you use that feature, chances are you would set the property yourself rather than relying on the default.

Second example: In older versions, [tt]SCATTER MEMVAR NAME <object name>[/tt] was permitted syntax. Now, it (rightly) gives an error. But if you were using that syntax previously, the results would have been unpredictable (were values scattered to memory variables or an object?). So, your code wouldn't have worked, which means you probably weren't using that syntax.

Those are fairly typical of the sort of changes you might encounter. So, I'm not saying you definitely don't need to re-test the entire app, but the chances are good that you won't encounter many problems when you. (The big exception is in the behaviour of SQL commands, which has significantly changed, but you can avoid that by means of [tt]SET ENGINEBEHAVIOR[/tt], as I mentioned before).

I hope this is of some further help.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[TamarGranor]

One other reason you might want to go to VFP 9 to deal with webservices. VFP 8 and 9 includes some tools that make working with webservices easier. Both the Task Pane Manager and the Toolbox (new tools that were added in VFP 8) give you the ability to register a web service. The Task Pane Manager also exposes a tool that lets you play with webservices to see how they work.

IntelliSense (VFP 7 and later) lets you easily set up the code to use a webservice.

Tamar

 

[KarenLloyd]

Hi Mike

Thank you very much for providing so much extra information for me on the subject of VFP6/9, SOAP and XML.

I have already done some work with XML, extracting some very simple XML anyway, I can build on this if I need to.

I don't have access to VFP 9.0 at the moment either. I may be able to buy a copy still, but that goes beyond my shoestring budget at the moment. I shall have to persevere with 6.0 for now.

I appreciate your extra input and will refer back to this if I can eventually upgrade.

Thank you

Best wishes

Karen

 

[Mike Lewis]

You're very welcome, Karen. Be sure to come back if and when you have more questions.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[KarenLloyd]

Hi Tamar

Thanks for the additional information regarding later versions of VFP and web services. I would like to be able to look into VFP 9.0 when I can - but I don't think it is going to be possible as part of this project.

Either way, I appreciate your input.

Thank you

Best wishes

Karen

 

[mplaza]

If you don't want to ( or can't ) use the soap toolkit, you can use plain http requests.

You only need to discover the soap envelope and soap action parameters, wich can be done using
a free software called soapUi.

Back in 2012 in foxite I posted a short guide on how to do it. Check:

http://www.foxite.com/archives/0000350247.htm

Marco

 

[KarenLloyd]

Hi Marco.

I do have access to SoapUI and an API guide with the actions and parameters I need.

Your link to your http example has just given me the huge boost I needed to kick start this week.

Thank you so very much. You are a superstar!

Best wishes

Karen

 

[KarenLloyd]

Hi again, you wonderful people

I just wanted to add something into here in case it helps anyone else.

The SOAP messaging is working just fine via HTTP as suggested by Marco (mplaza).

I will be dealing with XML data in VFP6 and I came across this today - How To Move Data from an XML Document into a FoxPro Table

http://support.microsoft.com/KB/253713

I can manipulate the Soap response to extract the sections I want for data and attachments, making sure to keep the Basenames. And having created tables that match the XML structure, this works perfectly so far. (I changed the code slightly to shorten any NodeNames over 10 characters)

I also have to deal with image data for photos and signatures from base64 encoded text. This article helped a great deal -> How to perform Base64 encoding/decoding using Cryptography API Functions

http://www.news2news.com/vfp/?example=88

Maybe these pointers will help somebody else.

Thank you once again for your support.

Best wishes

Karen

 

[mplaza]

Hi Karen good to know you succeed.

Marco

Ps:
For Base64 string encode/decode , check for vfp's strconv( string,13 ) / strconv( string, 14 ) ;-)

 

[Olaf Doschke]

Marco,

good point, but I think these strconv options were not available in VFP6. It's always hard to remember when a certain feature was introduced to VFP, even just a new parameter or parameter vlaue to the already existing functions.

Karen, with VFP6 it's much easier to make use of the wwXML library.
Download:

https://www.west-wind.com/wwxml.asp

Overview of high level (easy) methods:

https://www.west-wind.com/webconnection/wwClient_docs/_s8x02fe36.htm

Try the XMLToCursor and CursorToXML methods. They won't work with any XML, but the simple XML similar to the KB article should work.

Bye, Olaf.

 

[Mike Lewis]

I think these strconv options were not available in VFP6. It's always hard to remember when a certain feature was introduced to VFP, even just a new parameter or parameter vlaue to the already existing functions.

In general, a good place to look for version differences in functions is the Wiki version concordance.

That says that STRCONV() was present in VFP 3.0, but new settings and paramaters were added in most later version. Unicode and UTF-8 conversion was added in 7.0. The Base64 conversion that Marco mentioned was added in 9.0.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Olaf Doschke]

Just out of curiosity I have taken the code from [URL unfurl="true"]http://support.microsoft.com/KB/253713[/url] and tested it against XMLtoCursor and a self written routine using ALINES and STREXTERACT.

The MS KB solution doesn't take much longer than XMLTOCURSOR. Mainly because behind the scenes XMLTOCURSOR also uses msxml ole classes to parse (and verify) the XML and put it into an object model, which then is converted, AFAIK.

What I then did is used STREXTRACT and it's blazingly much faster.

I test with a rather medium sized 9000 record XML string and a simple record structure of three elements (row,col,val). So any xml record looks like <cell><row>1</row><col>1</col><val>A1</val></cell>. so eaqch <cell>...</cell> is a record. The 9000 records resemble the filled cells of an excel sheet, but that doesn't matter in this context. More important using either KB code or XMLTOCURSOR, the XML conversion takes about 3 seconds. In contrast ALINES and a loop on these lines using STREXTRACT, to get the inner xml element values, merely takes split seconds (0.05 s)! If that difference of getting the result data 60 times faster (not 60%, 60x) is interesting, read on:



This code idea even is checked with VFP7, might also even do in VFP6:

Create Cursor curResult(row I, col I, val M)
For liCount = 1 To ALines(laLines,Strtran(lcXML,"</cell>",Chr(0)),.T.,Chr(0))-1
  lnrow = VAL(StrExtract(laLines[liCount],'<row>','</row>'))
  lncol = VAL(StrExtract(laLines[liCount],'<col>','</col>'))
  lcval = StrExtract(laLines[liCount],'<val>','</val>')
  Insert Into curResult VALUES (lnrow, lncol, lcval)
EndFor

This is not general code, but you can get the idea from it. You have to adapt this to your specific situation, you could examine a sample XML file with the KB code to know the XML schema, or better the result cursor schema, but after a bit of coding for that structure and element names, the value extraction is much faster.

In more detail about this specific case:

Why Strtran(lcXML,"</cell>",Chr(0))?
</cell> here is the end of a record in the XML, that of course varies with each xml. The KB article takes all first level child nodes of the main XML node (everything from <xml> to </xml> as records, so it's delimited by whatever xml element names are for each record, thats working in general, but here you have to know the xml element names. In this case it's "<cell>......record xml......</cell>" per record, so each </cell> terminates a record.

Why ALines(laLines,Strtran(lcXML,"</cell>",Chr(0)),.T.,Chr(0)):
I convert each xml end element </cell> to chr(0), as the next up function ALINES will not take "</cell>" as line terminator in VFP7 or 6. You have to specify 1 character as line terminator. In VFP9 you can easier use "</cell>" as line terminator, and skip the STRTRAN completely. Chr(0) mostly is a non allowed char in XML files, therefore it's good to use as terminator char. Normal line breaks as CR and LF could be part of the data, eg memo texts, so taking normal line feeds would not work.

Why ALines(laLines,Strtran(lcXML,"</cell>",Chr(0)),.T.,Chr(0))-1:
The ALINES function converted each "line" of the lcXML into an array element of an laLines array, and each element therefore is a record. There is on exceptional element in the result array. The last element of laLines will only contain the closing "</xml>", therefore -1.

The first element laLines[1] will also have the opening <xml> element in it, but that won't matter, you use STREXTRACT now to extract the value of the inner single field xml elements named by the field names of the data. STREXTRACT is very fast in extracting these inner strings. STREXTRACT could also be used to extract everything between "<cell>" and "</cell>", but overall it's faster to extract all records with ALINES instead, as that will parse the whole xml string in one go and not parse the whole xml string reccount (9000) times.

Like with the KB article code you always extract text type values of course, but you can transform them to numbers by VAL, or dates by CTOD, etc., as I did or lnrow and lncol.

Bye, Olaf.