Mono WebClient encoding issue

Question

I'm trying to port a .NET application from Windows to Mono, but certain code that was working on Windows is no longer working (as expected) on mono:

WebClient client = new WebClient ();
Console.WriteLine (client.DownloadString("http://www.maxima.fm/51Chart/"));

it seems to detect correctly the encoding as UTF-8 (and manually setting the encoding to UTF-8 or ASCII don't work either) there are still '?' characters

Answer 1

You are writing to the console. Maybe your console is not configured properly to show certain characters. Make sure by debugging and storing the result into an intermediary variable.

Also the site you gave as example is completely messed up. The web server sends Content-Type: text/html; charset=iso-8859-1 HTTP header and in the resulting HTML you see <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> which of course is completely incoherent. You cannot expect an HTTP client to behave correctly when confronted to non-standard site, what you get is unexpected behavior.

Try testing on some web site that respects a minimum of web standards.

Remark: WebClient implements IDisposable, so make sure you wrap it in a using statement.

---

UPDATE:

To make it work with this particular site you may try downloading the response manually and specifying the correct encoding:

// You may try different encodings here (for me it worked with iso-8859-1)
var encoding = Encoding.GetEncoding("iso-8859-1");
using (var client = new WebClient())
{
    using (var stream = client.OpenRead("http://www.maxima.fm/51Chart/"))
    using (var reader = new StreamReader(stream, encoding))
    {
        var result = reader.ReadToEnd();
        Console.WriteLine(result);
    }
}

Answer 2

 using (var client = new WebClient())
        {
            client.Encoding = Encoding.UTF8;
           Console.WriteLine (client.DownloadString("http://www.maxima.fm/51Chart/"));
        }