Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
duplicates - server job
- Thread starter q4s72534
- Start date
You've got a few ways of handling duplicates.
- You can use an aggregator stage to group the data and remove duplicates.
- You can put a copy of the data in a hash file and perform a lookup to identify duplicates.
- You can purchase QualityStage which has a comprehensive set of matching functions.
- You can use the changed data detection CRC32 function to do a fast comparison of a large number of columns. There is an example uploaded on Ascential's DeveloperNet at - If your data source is a database or ODBC stage you can remove duplicates by adding a where clause or a group by clause in the select statement.
- You can use an aggregator stage to group the data and remove duplicates.
- You can put a copy of the data in a hash file and perform a lookup to identify duplicates.
- You can purchase QualityStage which has a comprehensive set of matching functions.
- You can use the changed data detection CRC32 function to do a fast comparison of a large number of columns. There is an example uploaded on Ascential's DeveloperNet at - If your data source is a database or ODBC stage you can remove duplicates by adding a where clause or a group by clause in the select statement.
- Status
Similar threads
- Locked
- Question
- Locked
- Question
- Locked
- Question