Screen Scrape Dynamic Content 1

[jaylrob]

I have a screen scraping .aspx file that works perfectly with plain HTML pages. However, if a page contains dynamic content (from an asp page), the dynamic content is not captured. I am using WebRequest and StreamReader to capture the html.

Any direction appreciated.

 

[jmeckley]

what do you mean by

if a page contains dynamic content (from an asp page), the dynamic content is not captured.

the webrequest has no knowledge of how the response was rendered, only that it was. whether is html, jpeg, php, aspx, etc. doesn't matter. a physical file may be present at all it just depends on how the page is rendered. but as the client you have no knowledge of that.

Jason Meckley
Programmer
Specialty Bakers, Inc.

 

[jaylrob]

The asp file has a conditional which grabs database content. Even though that database content is displayed on screen, it is not captured through the WebRequest.

Dim webRequest__1 As System.Net.WebRequest = System.Net.WebRequest.Create(temp) where temp is the url.

Does WebRequest finish before the database processing finishes?

 

[jmeckley]

you need to get the response back from the request. (google on how to) once you have the response then you process the stream.

Jason Meckley
Programmer
Specialty Bakers, Inc.

 

[jaylrob]

Thanks.