Getting U+fffd/65533 instead of special character from Query String

Question

I have a C# .net web project that have a globalization tag set to:

Answer 1

i think your problem is in the flash, not the .net. it sends the special character in a weird way. try to urlencode the search string bevore you send it to the server.

Answer 2

    public string GetEncodedQueryString(string key)
    {
        string query = Request.QueryString[key];
        if (query != null)
            if (query.Contains((char)0xfffd))
                query = HttpUtility.ParseQueryString(Request.Url.Query, Encoding.GetEncoding("iso-8859-1"))[key];
        return query;
    }

Answer 3

It turns out that ActionScript 2.0 will send the URL encoded/escaped with UTF-8 while ActionScript 3.0 used ISO-8859-1. The way to solve this was to change the Request.Encoding value inside Global.asax if an encoding is specified in the URL:

void Application_BeginRequest(object sender, EventArgs e)
{
    HttpContext ctx = HttpContext.Current;

    // encoding specified?
    if (!String.IsNullOrEmpty(Request["encoding"]))
    {
        ctx.Request.ContentEncoding = System.Text.Encoding.GetEncoding(ctx.Request["encoding"]);
    }        
}

Could it be done differently?

Regards, nitech

Answer 4

The problem comes from the combination Firefox/Asp.Net. When you manually entered a URL in Firefox's address bar, if the url contains french or swedish characters, Firefox will encode the url with "ISO-8859-1" by default.

But when asp.net recieves such a url, it thinks that it's utf-8 encoded ... And encoded characters become "U+fffd". I couldn't find a way in asp.net to detect that the url is "ISO-8859-1". Request.Encoding is set to utf-8 ... :(

Several solutions exist :

• put <globalization requestEncoding="iso-8859-1" responseEncoding="iso-8859-1"/> in your Web.config. But your may comme with other problems, and your application won't be standard anymore (it will not work with languages like japanese) ... And anyway, I prefer using UTF-8 !

• go to about:config in Firefox and set the value of network.standard-url.encode-query-utf8 to true. It will now work for you (Firefox will encode all your url with utf-8). But not for anybody else ...

• The least worst solution I could come with was to handle this with code. If the default decoding didn't work, we reparse QueryString with iso8859-1 :

  string query = Request.QueryString["search"];
  if (query.Contains("%ufffd"))
      query = HttpUtility.ParseQueryString(Request.Url.Query, Encoding.GetEncoding("iso-8859-1"))["search"];
  query = HttpUtility.UrlDecode(query);
  

It works with hyperlinks and manually-entered url, in french, english, or japanese. But I don't know how it will handle other encodings like ISO8859-5 (russian) ...

Does anyone have a better solution ?

This solves only the problem of manually-entered url. In your hyperlinks, don't forget to encode url parameters with HttpUtility.UrlEncode on the server, or encodeURIComponent on the javascript code. And use HttpUtility.UrlDecode to decode it.

Answer 5

If the app is expecting the URL-encoded request to be based on UTF-8, the character "ø" should be "%C3%B8", not "%F8". Whatever function you're using to escape/encode that request, you probably need to pass it the name of the underlying character encoding, "UTF-8".