Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Andrzejek on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

flat file source with varying columns in a foreach loop

Status
Not open for further replies.

TeamGD

IS-IT--Management
Feb 18, 2001
69
AU
Hi all

I have numerous flat files (csv) that I want to use a Data Flow in a ForEach Loop Container to extract and then insert into a single SQL Server 2005 table. All the flat files have a number of columns that contain common data. The flat files also have a number of columns that are unique to each (or a group of) flat file(s). Note: a few of the flat files have every possible column.

I have created a SQL Server 2005 table that has every possible column. I created a Data Flow that maps every possible column from a flat file to the SQL Server table. I then added the Data Flow into a ForEach Loop Container. The data from the flat files with all possible columns are loaded into the SQL Server table perfectly. However the data from any flat file that does not have every possible column is not loaded into the SQL Server table perfectly. Data is loaded into incorrect columns and many rows are missing. Is there a way of overcoming this problem? I do not want to create a Data Flow for each combination of flat file columns.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top