LWP script problem, strange

[MrCBofBCinTX]

I use a script to pull prices out of some websites to save time and avoid all that visual crap. One site has changed their pricing to images using javascript. After studying the output through several browsers (firefox, lynx, elinks, curl, wget), I discovered something odd. There isn't any link to images, but those are streamed out through javascript. But when I save the page (complete option) with firefox, I get a nice HTML line with the prices in it!

<div class="pricePanel" style="text-align: center;"><div class="price"><div style="vertical-align: top; margin-top: 4px; margin-left: -1px; font-size: 14px;" class="inlineblock"><cufon style="width: 8px; height: 14px;" alt="$" class="cufon cufon-canvas"><canvas style="width: 19px; height: 14px; top: 1px; left: -2px;" height="14" width="19"></canvas><cufontext>$</cufontext></cufon></div><div style="font-weight: bold; vertical-align: top;" class="inlineblock"><cufon style="width: 25px; height: 22px;" [COLOR=red]alt="61"[/color] class="cufon cufon-canvas"><canvas style="width: 42px; height: 21px; top: 2px; left: -3px;" height="21" width="42"></canvas>[COLOR=blue]<cufontext>61</cufontext>[/color]</cufon></div><div style="vertical-align: top; margin-top: 4px; font-size: 14px;" class="inlineblock"><cufon style="width: 21px; height: 14px;" [COLOR=red]alt=".99"[/color] class="cufon cufon-canvas"><canvas style="width: 32px; height: 14px; top: 1px; left: -2px;" height="14" width="32"></canvas>[COLOR=blue]<cufontext>.99</cufontext>[/color]</cufon></div><div style="margin-top: 10px; margin-left: 4px; font-size: 0.6em;" class="inlineblock"><cufon style="width: 19px; height: 13.2px;" alt="ea." class="cufon cufon-canvas"><canvas style="width: 33px; height: 13px; top: 1px; left: -2px;" height="13" width="33"></canvas><cufontext>ea.</cufontext></cufon></div></div>           <button aria-disabled="false" role="button" sku="060430" inlist="0" uom="EA" onclick="addToList(this)" style="margin-top: 2px;" class="lean addbtn ui-button ui-widget ui-state-default ui-corner-all ui-button-text-icon-primary"><span class="ui-button-icon-primary ui-icon ui-icon-plus"></span><span class="ui-button-text">Add to List</span></button></div>

The price for this item is $61.99 and that appears as print and as alt information in this line.

I would like to obtain this useful line for my script but I don't see how this is being generated by firefox

 

[MillerH]

Firefox can process JavaScript which LWP cannot, and is being helpful by saving the generated html to the disk instead of the source.

I would suggest that you take a look at WWW::Mechanize::Firefox. The cookbook for this module contains the following question: Fetch a page to a file using Firefox. Using that you might be able to duplicate your results when using Firefox manually.

Then you just need parse using a regex or your preferred method like normal.

- Miller

 

[MrCBofBCinTX]

Thanks, hopefully that will solve the problem