I need everyone's input for a newbie, please read

[lifter10]

I've just started a new position and it's been a career change for me (went from accounting to database analyst). I'm moderately experienced with databases, but need some advice from everyone.


We have a database of potential people we would like to send mailings out to. About everyday, a listing of new contacts will come through and this listing can come in any form (Excel, Word, Access, etc). I've been givin the responsibility to query our database and check to see if anyone in the new listing I received has already been entered in our system (basically I don't want duplicates).

What does everyone feel would be my best method to find out if a person already exists in our database? There are a few caveats though, we use a GUID as the primary key and not all information is in the database (for example not everyone's SSN is in the system). The best I can hope for is a first name and last name, but then I have to deal with people having the same names.

Suggestions would be greatly appreciated!! Let me know if I should post in a different forum.

Thanks,

Chris

 

[johnherman]

There are many companies making large sums of money doing what you are proposing. Some examples are Ascential Quality Stage, Trillium Software, Group 1 Software, Dun & Bradstreet, etc.

Most of these softwares use a scoring system to determine matches. For instance, 10 points for a last name match, 8 points for an almost match, 10 points for a zip code match, etc with a score of over (say) 35 being a match.

Good Luck.

-------------------------
The trouble with doing something right the first time is that noboby appreciates how difficult it was.
- Steven Wright

 

[lifter10]

So, do you mean that people contract out this kind of work or they purchase the software that helps determine the matches? A point system sounds like a good plan, getting people onboard with that plan maybe a challenge.

 

[johnherman]

Yes, it is often contracted out. Ascential Quality Stage is one software product which can assist you with the project. It uses a scoring system. It is, however, expensive; in the $100k +/- range. There are probably more products in varying price ranges. Sometimes you can get a bargain. Sometimes you get what you pay for. Good Luck !!

-------------------------
The trouble with doing something right the first time is that noboby appreciates how difficult it was.
- Steven Wright

 

[monsky]

We have a software that we use for cross-matching and deduplication called 'ClearMatch'. We use it to do clean up of possible duplicate records as well as comparing a set of records against another database.

 

[lifter10]

monsky, that's exactly what I would like to do. I'll look into ClearMatch and see what it's about. I would like to be able to do this on my own, but it sounds like a huge undertaking on my part.