scrape data from multiple datasources(DB servers) into centralized DB

[lizok]

hello,
i need some help.
i have 8 DB servers that collect data. every 5 min i need to scrape those 8 DB tables into one centralized DB (9th server).

1. i need to be able to select only unique rows that have not been selected before
2. i need to have those 8 DB servers available for new inserts (can't lock the table completely)

Any suggestions on how to effectively perform this task?

thank you all in advanced

 

[Sarky78]

Hi,

On the 8 db servers i'd have a field that determins if the data has been "scraped" already. this field would be 0 by default - on insert - then when the data has been scraped it would be set to 1 upon sucessful completion of the scrape. then insert that data into the 9th db as you like. surround it with a try/catch then you'll know if there is a failure and can handle as necessary

Hope this helps!

Tony