Extract data in blocks of 15,000 from outside server to my SQL database

[thorny00]

I have been tasked with extracting data from an outside SQL server. The problem is that I am hitting their "live" database. They only want me to pull 15k records at a time. The connection is set up and working fine. I want to use a date, i.e. service_date to pull the records, WHERE service_date between 2018-01-01 and 2018-01-31. How do I set up the query to pull the next months data without having to go in and changing the date for each extract? All help is greatly appreciated.

 

[simian336]

A couple of questions here to help nail down what your look for.

You posted this in Microsoft SQL Server: Integration Services (SSIS) Forum so I am guessing you are planning on using SSIS?

15k at a time with some time interval in between?

Is there another column like ID that numbered?

Is this a one time thing or something that you will be doing frequently?

Thanks

Simi

 

[thorny00]

Hi Simi,

Yes, I am using SSIS to move the data.
I'm using a date to pull the records - i.e. DOS between 2018-08-01 and 2018-08-31
The ID won't work great.
I may be using this process over.

Thanks!
Bob