How to automatically download a file that is dynamically created

[tsyle]

Hey guys. I searched through this forum and was not able to find the answer to this.

I have an application that needs to automatically download a file that is created dynamically after a form post.

Here's my work flow:

1. I log into a website
2. I navigate to a certain page lets say settlementReport.do
3. I post some data to the web page
4. It dynamically creates a text file and pops up a message box to download the text file

What I need is to automatically download the file or even better, stream the text file to memory.

Do you think this is possible?

Any help will be greatly appreciated.

 

[jmeckley]

It dynamically creates a text file

but once created the file exists. so read the file to memory and send to the user.

Jason Meckley
Programmer
Specialty Bakers, Inc.

faq855-7190
faq732-7259

 

[phyrtech]

tsyle,

To answer your question fully, I will need some additional information.

1.) Do you need your program to "log in" to the website from which you will be downloading this file?
2.) Will the program be posting data to the site in order to generate these reports?

If you are simply trying to download a file from a website without any sort of interactive feature (aka log in to the website), then you will likely want to look into the System.Net.HttpWebRequest or System.Net.WebClient classes:

[URL unfurl="true"]http://www.csharpfriends.com/Articles/getArticle.aspx?articleID=115[/url]

If you need your program to log in to the website to generate the report first, then there are a number of considerations:

1.) Security. How will you maintain security across your program AND the website? Will you use SSL to encrypt the HTTP session? How will the username and password for the website be stored locally? Will the user have to enter their credentials every time they want to generate a report?
2.) Handling of cookies and scripting sessions. WebClient and HttpWebRequest do have the ability to handle cookies if you're willing to go to the trouble to use them.
3.) Reverse-engineering the website. In order to log in and properly store the authentication cookies (assuming the website does it this way), you first need to know exactly how the website works. Does the website store a session id in a cookie on your computer?
4.) Scraping websites. It is generally not recommended to scrape websites in order to process information. If the website administrator makes a few minor modifications to the website, will your program be robust enough to continue working without recoding? What if they administrator changes the login mechanism?
5.) There are a number of other considerations that may arise depending on the specifics of the implementation.

In short, simply downloading a file from a predictable or static URL is fairly simple to do using the WebClient and/or HttpWebRequest classes. Constructing a program to log in and scrape a website for information is, however, a considerably more difficult task and is highly prone to scalability and maintainability problems.

You should check with the site administrator to see if they offer a programming API to interface with their website (which most websites do not offer, unfortunately). API's are (supposedly) robust and will not break when they update their systems.

Sincerely,
Nathan Davis

Phyrstorm Technologies, Inc.

http://phyrtech.com

 

[tsyle]

Hi guys,

Thanks for the reply.

Here is how my application is currently setup:

I have a web browser control that sends a post to a website to log into it. Yes this is probably a bad way of doing things because any changes on the website will require that I make changes to the code, but this is currently my only option.

After the page load, I navigate to a different URL until I reach the page with the button to generate the file.

I can then send a post to have the file generated but then a message box pops up asking me to download the file.

This is where I'm lost. I have researched using the WebClient and HttpWebRequest but was not able to keep my session when I moved to another page so I just went with the webbrowser control instead.

Would I be able to get the session and cookies from the web browser control and move it to a HttpWebRequest? And even if I am able to do that, does it matter that the file that gets generated has the following in the header?

Content-disposition: attachment; filename=fname.ext

This is more than likely the reason why the File Download dialog box keeps coming up but I have not been able to work around this problem yet.

Thanks for all the suggestions so far!

 

[phyrtech]

tsyle,

In order to support sessions, you will need to handle cookies which store the session id. You will need to extract the cookies after each request is made and then repost them to the web server for each subsequent request:

from:

http://forums.asp.net/t/945319.aspx

HttpWebRequst request = WebClient.Create([URL unfurl="true"]http://localhost/app/1.hd);[/URL]

request.CookieContainer = new CookieContainer();

request.UserAgent = "IE 6.0";

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

CookieCollection ckc = response.Cookies;
Cookie ck = ckc["ASP.NET_SessionId"];
if (null != ck)
{
   Console.Write(ck.Value);
}

This should give you some idea on how to access and store cookies between HTTP requests.

Sincerely,
Nathan Davis

Phyrstorm Technologies, Inc.

http://phyrtech.com

 

[phinoppix]

This post is more appropriate on the ASP.NET forums