getting corrupt zip and gz files with HttpWebRequest &HttpWebResponse 1

[DarwinIT]

I have used this code other places with no problem. The site I'm trying to download from now is always coming back as corrupt. Is there something else I need to add to this code to successfully bring this file down to my machine? Here's my code.



string strVars = "";

try
{
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create(string.Format("{0}{1}", strPage, strVars));
string[] h;
if (headers.Length > 0)
{
h = headers.Split(Convert.ToChar(","));
foreach (string s in h)
{
WebReq.Headers.Add(s);
}
}


WebReq.Method = "GET";

HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();

Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
StreamWriter sw = new StreamWriter(fileName,false);
sw.WriteLine(_Answer.ReadToEnd());

sw.Close();

 

[jmeckley]

I have used this code other places with no problem. The site I'm trying to download from now is always coming back as corrupt.

if the code hasn't changed, then the code isn't the problem.
have the server's changed?
are the servers configured differently?
has the file changed?
are you sure the file is not corrupt?
what headers are passed with the response?

Jason Meckley
Programmer

faq855-7190
faq732-7259

 

[DarwinIT]

Server on the other end is not one we have ever downloaded from before. If I browse to the site and click on the dropdown and the button for download - the file is saved to my machine just fine.
The corrupt version is about a million bytes (almost double) bigger than the one I downloaded manually. Looks like perhaps a problem with the character set?? I am not sending headers to them. I'm getting back

{Content-disposition: inline; filename="IpToCountry.csv.gz"
Connection: close
Content-Length: 1290205
Content-Type: application/octet-stream
Date: Thu, 01 Sep 2011 19:37:14 GMT
Server: Apache

}

The content length shows the number of bytes I should be storing. Perhaps I just need to alter my StreamWriter to only write out that number of bytes or replace the ReadToEnd with Read and the number of bytes I want?

 

[DarwinIT]

I think this is what's going on.
I'm seeing a ton of EFBFBD sequences. I would conjecture that adding 2 characters for each high ascii character would swell the file up considerably with a zip file.

http://social.msdn.microsoft.com/Fo.../thread/21b2c560-ed16-4edb-bdab-6a55902d6ae3/

 

[DarwinIT]

I had to do two things to get this to download successfully. I added encoding with the value of "28591", which is the .NET reference for ISO 8859-1.

And i had to change the WriteLine on the StreamWriter to just Write to avoid add a LineFeed CarriageReturn at the end which apparently corrupts the file.

 

[whosrdaddy]

I think that's the reason you need to use the binarywriter class.
Streamwriter is used for strings (and hence the encoding problems)

http://msdn.microsoft.com/en-us/library/system.io.binarywriter.aspx

/Daddy

-----------------------------------------------------
What You See Is What You Get
Never underestimate tha powah of tha google!

 

[DarwinIT]

Thanks Jason and Daddy for your feedback!!!

 

[jmeckley]

thanks for sharing the solution. hopefully this will help others as well.

Jason Meckley
Programmer

faq855-7190
faq732-7259