XMLTOCursor problems

[EzLogic]

VFP9 SP2
Various machines.
Tables in DBC files

I have an App (POS application), that syncs the dbf files with our corporate server via FoxWeb.

the way i sync is this:
i take the records that are marked to be sync'd and i convert them to xml:

lcfile = SYS(2015)+".xml"
CURSORTOXML(lcCursor, lcfile, 1, 512, 0, "1")

on the server side, i get the xml post and i conver them from xmlto cursor
i get the xml from the request.form('xml') and i filetostr it to a file etc..
then:

xmltocursor(lcFile,'ToSync',512)

this code fails alot since some of the records have some wierd characters from data entry people. like chr(2) or chr(13) or chr(9) ..

is there a way to make it work and strip out wierd characters?

i did:

lcXML = strtran(lcXML,chr(2),'')
lcXML = strtran(lcXML,chr(9),'')
lcXML = strtran(lcXML,chr(0),'')

but, behold, some other wierd character will show up unexpectedly once in a while,....

thoughts?


Ali Koumaiha
TeknoSoft Inc.
Michigan

 

[Mike Lewis]

Well, the first step is to define "wierd characters". Let's assume the term means all characters whose ASCII code is less than 32 (that is, all "unprintable" characters"), except of course CHR(10) and CHR(13).

In that case, you could so something like this:

lcWierd = CHR(0) + CHR(1) + CHR(2) + ..... + CHR(31)
  && but leave out 10 and 13
lcXML = CHRTRAN(lcXML, lcWierd, "")

That should do it, unless someone can suggest a slicker solution.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy

 

[EzLogic]

so this code will work?

  for x = 1 to 31
    if inlist(x,10,13)
       loop
    endif 
     lcXML = strtran(lcXML,chr(x),'') 
  endfor

Ali Koumaiha
TeknoSoft Inc.
Michigan

 

[Mike Lewis]

Yes, that should do it. (Assuming you agree with my definition of "wierd characters".)

I think my code would run faster, but perhaps at the cost of more initial typing.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy

 

[Olaf Doschke]

You could check the XML after creating it via XMLAdapters LoadXML method and catch error 2110, to see what is wrong. Atually report about wrong chracters can also indicate other problems, eg:

o = CreateObject("xmladapter")
* next line validates
o.LoadXML("<xml></xml>",.f.,.t.)
* next line does error, which points out first closing ">" as wrong character of a name. 
o.LoadXML("<xml b></xml>",.f.,.t.)

The error rather should point to the b or state the closing > comes too early. What is allowed within the xml start tag are some additional attributes, but not b, and even if b would be an allowed attribute, it would nneed to continue with = value, before the closing > would be allowed.

So in short, this is not only about the ascii codes, it can also be about the position of wrong characters. Some thing that may hurt is having > in the data, but I assume vfp changes those to > when outputting to XML.

Overall Foxpros cursortoxml creates correct xml in itself, so the culprit could really be some chars in your char fields, that don't belong to codepage 1252. Fixing your data rather than the resulting xml should be done. I can only assume such chars come in via scanner of your POS system or anything that does not only put text into your char fields. But chr(9) is not weird, it's simply a TAB, chr(13) is a carriage return, also not weird.

You should rather get to the ground of what characters are causing trouble, instead of simply removing low ascii codes. You may have ESC or other codes typical for low level adressing of printers in your data. Then change these fields to char (binary) or varbinary and they will be base64 encoded in the XML to not hurt and still arrive untouched at the other side.

Bye, Olaf.

 

[Olaf Doschke]

To add a bit of info: Indeed I just assume codepage 1252, but no matter if that is your real codepage or some other one, typically they all allow one-byte characters from chr(0) to chr(255), so any value is allowed and you can indeed also store binary data in nonbinary field types in foxpro because of that.

It's just XML, that doesn't allow control characters 0-31 and also 0x7f = 127 should be avoided. More precisely, 0x09, 0x0a and 0x0d are allowed, also. Some other chars need to be escaped, eg < and > must be encoded as &lt; and &gt;, even though they are not in that range of control characters, but because they would break the xml structure, obviously.

So there really is a mismatch of the interpretation of what a codepage is made up of and what you can transport via XML.

Espacing the control characters 0x00-0x1F does still not comply to the XML specs and so you really need to avoid them, despite moving over to binary fields, which can be embedded as base64 encoded in the XML.

Bye, Olaf.

 

[EzLogic]

thanks Guys.

Olaf, its hard for me to control the data on the client side. its a distributed application (POS) and their headquarter runnig foxweb.

so, when i get the xml, i have to manipulate it. its not very easy to keep deploying updates to the stores (sites)

Ali Koumaiha
TeknoSoft Inc.
Michigan