Wikimedia .jar file..

[youradds]

Hi,

Not really sure if anyone will have any ideas.. but I thought it was worth asking =)

I'm trying to import the Wikipedia dumps from

http://download.wikimedia.org/tools/README.txt.

However, I'm having problems with the "limits" in place.

For example, using this command;


wget

http://download.wikimedia.org/wikipedia/en/20051127_pages_articles.xml.bz2

java -server -jar mwdumper.jar --format=sql:1.5 20051127_pages_articles.xml.bz2 > dump.sql

..gives;

user@undevmac wiki $ java -server -jar mwdumper.jar --format=sql:1.5 20051127_pages_articles.xml.bz2 > dump.sql
1,000 pages (231.803/sec), 1,000 revs (231.803/sec)
2,000 pages (260.146/sec), 2,000 revs (260.146/sec)
Exception in thread "main" java.io.IOException: Parser has reached the entity expansion limit "64,000" set by the Application.
at org.mediawiki.importer.XmlDumpReader.readDump(Unknown Source)
at org.mediawiki.dumper.Dumper.main(Unknown Source)
user@undevmac wiki $

Anyone had any experience with this, and how to solve/raise the limit?

TIA!

Andy

 

[sedj]

30 seconds on google :

http://java.sun.com/j2se/1.5.0/docs/guide/xml/jaxp/JAXP-Compatibility_150.html

To set the entity expansion limit using the system property, use an option like the following on the java command line: -DentityExpansionLimit=100000
[/code]



--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[youradds]

OMG.. you're a star! I did read about that, but tryied putting it in the wrong place *blush*

i.e;

java -server -jar mwdumper.jar -DentityExpansionLimit=100000 --format=sql:1.5 20051127_pages_articles.xml.bz2 > dump.sql


..instead of ;

java -server -jar -DentityExpansionLimit=100000 mwdumper.jar --format=sql:1.5 20051127_pages_articles.xml.bz2 > dump.sql

I'm a perl/MySQL type of person.. so this JAVA stuff is all very new to me (in fact, this is the first JAVA application I've ever tried using from command line =))

Thanks again