HTML source code?

[rvBasic]

I used (and still like) to scrape stock information from some web pages. For instance, I use(d) the NYSE Euronext pages to extract Option prices. I could find that information by displaying the HTML source and by comparing it to the actual displayed page. I then could locate some "markers" which I used in a VBA program to automate the extraction.

But ... lately they improved the website's look and feel, and since then I'm unable to run my program. When I now look at the source code, it seems no longer to reflect the displayed page. How is this possible?

If for instance you look at the page [link

http://www.euronext.com/trader/summ...ciceType=USA&mnemo=BP&selectedMepDerivative=7

]BP Option Price[/url] you clearly see the word Classification. However if you display its source code and perform a search for that word, no matches will be found.

_________________________________
In theory, there is no difference between theory and practice. In practice, there is. [attributed to Yogi Berra]

 

[MakeItSo]

This will always be a risk.
If you update now, you might encounter the same problem again in a few weeks / months.

Have you thought about using their RSS feed to get that information?
Here you find some of their XML live feeds:

http://www.nyse.com/tradingsolutions/marketdata/datainformation.html#Realtime Datafeeds

RSS feeds:

http://www.nyse.com/about/newsevents/1149674941598.html

Good luck!

[navy]"We had to turn off that service to comply with the CDA Bill."[/navy]
- The Bastard Operator From Hell

 

[ChrisHunt]

The actual stock prices are displayed on another page, via an iframe on the page you linked to - in this case it was

http://euronext.nyx.com/EN/derivatives?Class_type=O&Class_symbol=BP&Class_exchange=DLON

However, I suggest - for both ethical and practical reasons - that you look for a proper API that you can use for stock prices, rather than scraping other peoples' sites.

-- Chris Hunt
Webmaster & Tragedian
Extra Connections Ltd

 

[rvBasic]

Thanks for taking time to investigate my question. Probably NYSE Liffe Equity Derivatives Market Summaries could satisfy my request in a more structured way. But its price (€175/month) looks prohibitive to me.
Perhaps I have to add that I am in no way a professional user and I have no intention to write professional software. I certainly do not need life streaming data. From my perspective it is no different than looking at the data, taking notice and entering them in a database. But yes, it is somewhat faster. However I do take notice of your advice and if I can find a suitable and affordable API, I certainly will investigate it.

But back to the question: do I understand you right that if a webpage is subdivided into iFrames and I ask the for the source code of the Webpage, then the iFrame's HTML will not be displayed?
And also, didn't I read somewhere that iFrames would be demoted?

_________________________________
In theory, there is no difference between theory and practice. In practice, there is. [attributed to Yogi Berra]

 

[johnwm]

If you want to scrape an iframe, you need to read the iframe rather than the container page. Follow the link that leads to the iframe and you will find the data.
However you should first carefully read the "Terms of Use" page on that site, which appears to specifically forbid scraping.

If you want the best response to a question, please check out FAQ222-2244 first.
'If we're supposed to work in Hex, why have we only got A fingers?'
Drive a Steam Roller

 

[rvBasic]

Message got, Project cancelled. Manual interface will do.
Thanks to all for time and effort.

_________________________________
In theory, there is no difference between theory and practice. In practice, there is. [attributed to Yogi Berra]