Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

ETL on SAP BEX report - missing fields

Status
Not open for further replies.

AndrewZX

Technical User
Oct 22, 2013
2
I've done basic table loads with SSIS and had success but now have a complicated report to load.
The format is similar to this:
Code:
TransactionID UserID  firstname lastname itemID itemdescription itemcost
100001        1001    Joe       Jones     abc    black shoes    $50.
100002        1002    Jane      Johnson   bcd    red shoes      $60.
                                          cde    green shoes    $61.
                                          def    blue shoes     $60.
100003        1003    Jay       Jackson   efg    orange shoes   $50.

The problem is to impute or fill in fields missing from the report, to fill in all fields
during the ETL. This would be a simple programming task in Perl or Python, but how
do you approach this in SSIS?

Any pointers or recommendations greatly appreciated!
 
Where does this missing data come from?

----------------------------------------

Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind - Bernard Baruch

Computer Science is no more about computers than astronomy is about telescopes - EW Dijkstra
----------------------------------------
 
I'm assuming the missing data is from the previous line on the report that has those filled.

Pure SSIS - don't advise it if volumes are big.

Load the file to a recordset object

On a dataflow add script as source, and on this script you read the recordset above in a loop, storing the values of each row and sending the values of the previous row to the output collection if they are empty

mix of SSIS/T-SQL.

Load the file onto a staging table adding a row number while doing it (either identity column or a row generator on SSIS)
Then using t-sql perform a update of the rows with the required

Regards

Frederico Fonseca
SysSoft Integrated Ltd

FAQ219-2884
FAQ181-2886
 
Frederico,

Thanks, yes missing fields come from a previous record. OK, I will look at recordset object + T-SQL.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top