ProtocolError with HttpWebRequest

[RileyCat]

I am attempting to automate the download of files from a secured web site. I have been able to automate the upload process but I am being plagued by errors when I attempt to automate the download process. Here are the details . . .

C#; Using HttpWebRequest
I am able to successfully log on to the web site; I am able to successfully get a response back with a directory listing of the files that are available for download.

I have formatted my query string exactly as it appears on the web source when viewed through a browser. However, when I try to perform the download action via my C# application, I receive the following error . . . (I'll post as many pieces that I think might help debug this . . .)

- System.InvalidOperationException {System.Net.WebException} System.InvalidOperationException

+ System.SystemException {"The remote server returned an error: (500) Internal Server Error."} System.SystemException

StackTrace " at System.Net.HttpWebRequest.CheckFinalStatus()\r\n at System.Net.HttpWebRequest.EndGetResponse"

Here's what my code looks like . . .

				postdata = postdata.Substring(0,postdata.Length-1);
				data.Initialize();
				data = Encoding.ASCII.GetBytes(postdata);
				string DowloadURL = ServletURL + postdata;
				webRequest = (HttpWebRequest) WebRequest.Create(DowloadURL);
				webRequest.CookieContainer  = cookies;
				webRequest.Method = "POST";		
				webRequest.ContentType="application/x-[URL unfurl="true"]www-form-urlencoded";[/URL]
				webRequest.ContentLength = data.Length;
				requestWriter = webRequest.GetRequestStream();
				if (requestWriter.CanWrite)
				{
					requestWriter.Write(data, 0, data.Length);
					requestWriter.Close();
				}

				responce =  (HttpWebResponse) webRequest.GetResponse();

				s = responce.GetResponseStream();
				sr = new StreamReader(s);
				stext = sr.ReadToEnd();

I have this surrounded by a try/catch block and the exception is being thrown when I attempt to do any sort of read on the response stream.

Any ideas, suggestions, comments, or debug hints would be most appreciated!

Thanks in advance!

Stay Cool Ya'll!

-- Kristin

 

[onpnt]

A solution was found to this problem and thanks goes to chrissie1

The solution?!?

open the *.exe.config for your application, or the file for your entire .NET system

Look for the following XML tags in the configuration file

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing = "true"/>
</settings>
</system.net>

if the system.net tag is not there, then add it.
Add the setting to allow unsafe headers as shown.

In addition to create the app.config file perform the following

Within Solution Explorer, right-click <appname>, point to Add, and then click Add New Item.
In the Templates list, click Text File and name the file app.config.
Configuration files with the name app.config are automatically copied by
Visual Studio .NET as part of the build process to the output folder
(for example, <projectdir>\bin\debug) and renamed as <applicationname>.config.

Click OK to add the new configuration file.
Add the following configuration elements to the new configuration file.

Compliments of researching
MSDN @ Building Secure ASP.NET Applications: Authentication, Authorization, and Secure Communication

My 2 bits; I would do quite a bit of research on this. Security is a concern I thought mainly in the wrong hands and not a stable place you are pulling from.

Any additions welcome or corrections etc.

 

[chrissie1]

I had to use it for a camera that was sending the wrong headers so no big deal there. The camera (connected to a microscope) has a built in webserver so I couldn't change that. And the camera is for internal use only so no security risks.

Christiaan Baes
Belgium

"Time for a new sig." - Me

 

[chrissie1]

And I forgot to mention that they changed this since 1.1 sp1. And thanks to MS for not telling me.

Christiaan Baes
Belgium

"Time for a new sig." - Me