MassLoader

[kjmagyor]

I am having performance problems running MassLoader. I am trying to move about a million rows into a table. Catalog Manager is on an NT server and the database is on an AIX box.

The first thing I did was to run MassLoader with the LOAD option, but I got errors (can't remember exactly what they were, but I can recreate them if I need to). I was told that I cannot use the LOAD option with MassLoader if catalog manager is on a different server than the database. So, I switched to the SQLIMPORT option. Now it is running VERY slowly. I submitted the load yesterday before I left for the evening, and about 4,000 records/minute were being loaded This morning, when I got in, only 150,000 rows were loaded, and the rate is now about 400/minute.

I have removed all of my indexes from the tables for better performance, but it didn't seem to help.

My questions:
*Has anyone else had this type of thing happen? What did you do to fix it?
*Can anyone verify that catalog manager DOES need to be present on the same box to allow the LOAD option on MassLoader?
*Could I be having network problems since the file and catalog manager are on different machines?
*What can I check to see what is hanging this?

Any help would be greatly appreciated. Thanks!!

 

[gregsimpson]

kjmagyor,

LOAD utilities of ten work very inefficently if the data your loading is not in the Clustering Index order. Is this the case with your load?

Cheers
Greg

 

[kjmagyor]

As I look over the post, I realize that I was incomplete in my information. I forgot to mention that this is a migration of WebSphere Commerce 4.2 to 5.4 and that it is the member subsystem that I am trying to migrate.

Greg, thanks for your reply, and that could be the case. I will check tomorrow to see if that is the case.

Thanks,
Kathy