Capturing data from multiple web pages at one time

[chris86t]

I have made a program that captures data from a webpage. This action is repeated a few hundred to a few thousand times. Right now it is taking about a second per capture because of the time it takes to load each page. Here is an outline of what I'm doing.

1) use web client to retrieve the page
2) loop through html until regular expression match is found
2) save data to an array
4) do it again.

How would you make this process any faster?

 

[SkipVought]

...and your code?

How could anyone evaluate this process without evaluating the star or villain of the process?

Skip,
[sub]
Just traded in my old subtlety...
for a NUANCE![/sub]

 

[ug505]

I don't think you can make it any faster unless you get faster internet. You could try having multiple webbrowser components capturing the data.