Huge XML/XSLT Transformation

[bluebytez]

Hi there,
I hope someone can help me out with a problem I've been facing. I have an XML file which can potentially grow up to thousands of records. I would expect this to slow down the transformation tremendously. Also, I would like to implement dynamic paging and sorting in this huge XML file, but everytime the XML is reloaded and transformed again, the whole file has to be loaded and this would make it a very slow process.
Anyone has any idea what is the best way for me to handle this ? Should I split the XML file into a few files with standard names or stick to one huge file ? Either way, what is the best way to implement dynamic sorting and pagination ?
Thanks !

 

[MrTom]

it depends how fast the rendering needs to be. even for a large file of 1000+ items it may only take 5 seconds on a reasonable machine.
sorting could slow things up tho and if there are only a few fields you can sort on then maybe it's worth precalculating the sorts into several different files - altho this depends on how often the file is updated.
the best option (in theory) would be an XML database or a regular DB and then you shouldn't have to worry about the speed any more.

 

[xso]

Hi there,

I agree with MrTom. We have a similar situation at the office and we extract the xml we need from a database. When we need to parse the full set of data (more than a 1000 records), it takes something like 5 seconds on a Sun Ultra 5 machine.

I hope this helps.

Cheers,

xso