Multiple ETL tools writing to DW

[aali123]

What are the consequences of using multiple ETL tools to write to a single data warehouse? Apart from rewriting error logic and resuable code, is there anything from the audit perspective? We are using an ETL tool for loading and planning another web-based application to write directly to some tables in the warehouse. Is it a sound idea? Thanks

 

[KingCrab]

Maintaining 2 systems.

Most tools out there should be able to do everything you want, why would you need two?

 

[aali123]

Yes, ETL is able to do the job, however, for some reason (time constraints, convenience) and other it has been decided to use a separate web-based tool. I am not in aposition to go into detail but can someone please outline possible consequences of using separate tool? Thanks

 

[johnherman]

Assuming you're stuck with the two ETL tools (agree with KingCrab that making such a choice voluntarily is bad), the only worry I would have is making sure each process is independent and doesn't trash the other in terms of CPU, temp space, audit tables, or worst case, overlapping data columns. Those would need careful analysis and troubleshooting when problems occur.

Two ETL processes not much more a problem than having two other applications, such as purchasing and inventory where the overlap is the only area for concern.

-------------------------
The trouble with doing something right the first time is that nobody appreciates how difficult it was - Steven Wright

 

[vmcburney]

It helps if you can schedule the jobs of both tools through the one scheduling tool, this lets you build some interdependence between the two loads. For example ETL tool 1 loads inventory tables and ETL tool 2 has to wait for those to finish before running purchasing loads. Automatic triggering between the two, where tool 2 only runs if tool 1 was successful, will make maintenance easier.