I have a SSIS package, part of a data warehouse ETL, that Reads in from 2 source tables approx.
src 1 = 350K rows.
src 2 = 120K rows
This are both sorted in the extrat and then merged together in a Left outer Join (merge join task). Based on specific column conditions in a conditional split, the data is output ti be handled in various ways. The first check filters out the Largest number of rows roughly 2 thirds of these and no additional processing is done. Now I am pretty sure that SSIS doesn't just flush these records from memory during the execution of this package. I am wondering if anyone has encountered a method that essentially takes this output and then flushes the records from memory.
Shoot Me! Shoot Me NOW!!!
- Daffy Duck
src 1 = 350K rows.
src 2 = 120K rows
This are both sorted in the extrat and then merged together in a Left outer Join (merge join task). Based on specific column conditions in a conditional split, the data is output ti be handled in various ways. The first check filters out the Largest number of rows roughly 2 thirds of these and no additional processing is done. Now I am pretty sure that SSIS doesn't just flush these records from memory during the execution of this package. I am wondering if anyone has encountered a method that essentially takes this output and then flushes the records from memory.
Shoot Me! Shoot Me NOW!!!
- Daffy Duck