Extract data with details from .ASP 2

[nja5150]

Hello-

I have access to a web driven report site that has a list of transactions. Each transaction has a detail link associated with it. The site has a built in export to excel for the list of transactions....but not for the extra detail you can link to on each transaction.

I need to pull this data out

The links for each transaction do run in an ascending order

so..

transaction 1 <link>

http://www.mysite.com/report/Item_Detail.asp?id=55468

transaction 2 <link>

http://www.mysite.com/report/Item_Detail.asp?id=55469

The detail info is text based and appears in a pop up. I would have to code something to detect HTML for fields.

Is there a program that can go in and extract all the linked data that I need for each transaction. There are too many to click manually and save.


Thanks for any help / recommendations / ideas

 

[Sheco]

You need to pull the data out but what do you need to do with it when you are done?

There are programs that will follow all the links and save everything as a local html file on your computer so you can browse it offline. Is that what you want?

Or are you looking to gather all the date so that you can aggregate it into a single file or database or spreadsheet or something?

Suppose you downloaded the Excel file... Is there something in the spreadsheet that you could use to accurately predict the URL of the details file? For example that ID in the QueryStrings you posted... is that ID in the excel file? If so you could use the Excel file to give yourself a list of all of the additional urls that you wouuld be interested in. It is much easier to parse an excel file than a load of HTML and it is probably a lot less likely to change formats.

Tell us more about the project.

 

[DotNetGnat]

Is this(web driven report site) your server or third party server...may be you need to use ServerXml to grab the links...

Do you need to grad just the id's or the whole links...i am not quite sure what you actually meant...

some examples should help...

-DNG

 

[nja5150]

This is a web driven report site. I do have to log in, and I do have full access to what I see on the screen.

Yes my goal is to put this information into a database. If that means two separate excel files--- transaction list, and transaction details for each transaction then that is okay. I can work with that.

The goal is to get this popup detail out to a separate table that I can relate to the transaction list. The transaction list is solved as I can easily export that. The developers didn't put in a fuction to dig the trasaction details. Punks!!!

Structure
----------

Transaction Table
-----------------
A1 transaction number (html link)
B1 membership number
C1 card type
D1 etc.
E1 etc.

All records in A1 have an HTML link.

Yes I can decipher the links. For instance the first transaction number HTML link is "

http://www.mysite.com/report/Item_Detail.asp?id=55468"

and each continuing "...asp?id=55469", ".asp?id=55470", etc. etc. The links do increase by + 1, they aren't random.

The transaction number is "1" in excel, but the HTML link is not ".asp?id=1" it's something like ".asp?id=55468". So I would have to get the first records link and use that for each following record....and just "walk" down the rows.

When I click A1, I get a popup. HTML text with call details like location, destination, notes, etc. This information isn't in table format like my transaction list. It's simple text separated by line breaks.

So the popup when clicking a transaction link says:

Trasaction Detail Popup (one for each Transaction ID)
-----------------------
Name: Fred
Location: My house
Destination: Your house
Note: Good luck working with this mess.

I know what I need to do...I can see it in my head, I just don't know how to get there / automate it.

Sheco - I can go either way. I think a program that will browse all the links and put them to HTML files would be a great start-- but yes lots of work. If it would save all the linked transaction detail by their transaction number (which is what I click on), then it will all be organized.

Is there then a program that can then say. Okay, each HTML file is a ROW/Record, and this HTML TAG is a field A, this HTML TAG is field B...etc. etc. then put that to an Excel file or Database file?

Quick and dirty could be to text edit find and replace HTML TAG using commas to generate a .CSV file I guess. Then mass import the separate .CSV's and merge them into one right?

I can get the data complete with HTML links into the Excel spreadsheet though, so maybe I don't need to go that route.

Yes I do want to aggregate

Yes I can predict the links. The first transaction link gives me the starting transaction ID (

http://..asp?id=55468)

and the number increases by + 1 for each transaction record after that.

Yes, I can get the data, with the HTML link addresses and all into Excel

Thanks for the quick reply guys!

Hope this helps you further.

I can supply you guys with an example of what I am working with if needed.