PowerCenter Platforms and Performance

[yaffle]

I have a general question regarding what platforms people are running Informatica on out there, and what sort of performance they are experiencing.
I'd be interested in the vendor, operating system, amount of processors and RAM, and the amount / type of data being transformed.

 

[geenew]

We are using a M80 Rs600 and the same versions of informatica. 4 CPU. 2 Gb memory. We use tables up to 7 miljon custommers and table with facts up to 200 Miljon.

We are still in the development fase. The performance is our bigest problem at the moment. Therefore i'm very interested in information about this topic.
I am looking forward to othere reactions to this question.

 

[yaffle]

geenew, a small proof of concept at my site with PowerCentre 5.1 using a 4 CPU NT box with 1Gb RAM showed high paging and disk usage - it was concluded that considerably more RAM may be needed as it is planned to run many streams in parallel. Did you specifically choose an IBM M80 for the job, or is it already your RDBMS platform ?

 

[walkerci]

There are many aspects to performance tuning ETL environments (CPU (32bit vs 64bit), RAM, disk configuration, network(interface, speed/loading, hops, etc), DBMS server capacity, data volumes, transform complexity, etc).

That is why many organizations need to bring in outside experts to help get them started.

Some general recommendations:

1. More RAM is usually a good thing. ETL tools like to cache data to reduce I/O. This also reduces paging.

2. ETL and RDBMS should be on separate boxes. These boxes should be on the same network segment, the higher the bandwith the better (consider 100 Meg Ethernet a minimum configuration), and be sure to use a switch (not a hub). I would also dedicate a separate network adapter to ETL on the RDBMS box.

3. RDBMS disk configuration for ETL / Data Warehousing.
I could write a book on this. There is way too much to cover in an email and this is very application/environment dependant.

4. Transform complexity / SQL optimization. More material for a book.

In general, you need to look for bottlenecks, try various approaches to mitigate, and measure the impact step by step.

I hope this helps.

Charles Walker

 

[djlocke]

Hi,

a few basic tips:

- you need about 1,5 CPUs per (unpartitioned) session
- memory requirements depend mainly on the size of lookups, joins, aggregations, sorts. Try to avoid disk caching by increasing the shared memory size parameters.
- avoid network bottlenecks, install your Informatica server and the target database on the same machine

Rgds.
Juergen