Data flow stops in lookup

[rflouw]

I have several lookups but the data flow stops at the fifth lookup without utilizing all the upstream data. This is a fairly basic flat file to database insert package in SSIS 2008. The lookup nodes lookup the corresponding surrogate keys before data is inserted into a fact like table.

I‘ve inspected the incoming data and the lookup data exists in the source. I’ve tried the cache connection manager. Same result.

The data flow simply dwindles down from 500K records to 5K.
Any ideas?

 

[MDXer]

Have you tried using an error output to see if it is bad data? if so there are a number of ways to handle bad or missing data.

 

[rflouw]

Tried the error output. There are no error output. All the data in the source exists in the lookup tables. It as if the package is running out of steam as soon as the lookups are hit.

I've also tried all the Cache modes, but the result remain the same. At the peak during package execution the 64 bit server has 2.5GB physical memory available.

 

[MDXer]

how much memory is being used? Remember that everything in the lookups is being cached in memory your dataflow is utilizing memory buffers as well. if you don't have enough memory it could be the issue. what does the output window report when you running it in debug?