• System.Net.HttpWebRequest spits out garbage.
    5 replies, posted
I have a method that gives me the source of a webpage: [code] public static string[] DownloadPage(string url) { StringBuilder sb = new StringBuilder(); byte[] buf = new byte[8192]; HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); //Console.WriteLine("Listening for response"); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); //Console.WriteLine("Recieved. Parsing."); Stream resStream = response.GetResponseStream(); string tempString = null; int count = 0; do { count = resStream.Read(buf, 0, buf.Length); if (count != 0) { tempString = Encoding.ASCII.GetString(buf, 0, count); sb.Append(tempString); } } while (count > 0); string result=sb.ToString(); string[] split = result.Split('\n'); return split; }[/code] But it usually spits out garbage like this: [img]http://www.wuala.com/en/api/preview/Hypershadsy/Images/webrequest_garbage.png[/img] And if I try to call the method again, it returns the EXACT same string[]. Weirder still, I try running the program a minute after it quits and returns the same string again. But it usually "fixes" itself if I wait three minutes afterwards. And by "fixes", I mean it spits out a different brand of garbage. Sometimes it works, though. Once every ten tries. Is this the work of a cache [del]or cookies[/del]? Is there a way to flush out the cache (or whatever makes it return the same garbage)? Is there a different method I can use?
Why make your own when you can use WebClient.DownloadString? [url]http://msdn.microsoft.com/en-us/library/system.net.webclient.downloadstring(v=VS.80).aspx[/url]
Still the same garbage. But that is a nifty class.
Ran your code multiple times for [url]http://www.google.com[/url] [img]http://i41.tinypic.com/dbn9u.png[/img] Not sure what's the problem? Can you give me the url you're using?
It IS just for the site I'm using. Doesn't matter now, I realized there was a program already out there that does the job of the program I was making. Thanks for the WebClient class.
It's because of HTTP transfer encoding. In this case, it appears that the response has been gzip'd
Sorry, you need to Log In to post a reply to this thread.