Special and accented characters

Question

I am doing some work for a French client and so need to deal with accented characters. But I\'m running into a lot of difficulty, I am hoping the solution is simple and that

Answer 1

EDIT: I have found that string are malformed when simply displaying the the request parameter from a form. (ie, request.getParameter("string") already has malformed content).

This can have three causes:

1. It's a GET request and the server isn't configured to use UTF-8 to parse request URI. It's unclear which server you're using, so here's a Tomcat-targeted answer as example: set URIEncoding attribute of the HTTP Connector in /conf/server.xml to UTF-8.

2. If it's a POST request, then you need to ensure that the servletcontainer uses UTF-8 to encode the request body. You can do that by request.setCharacterEncoding("UTF-8") beforehand.

3. The console which you're writing the parameter to doesn't support UTF-8. It's unclear which console you're talking about, so here's an Eclipse-targeted answer as example: in Window > Preferences > General > Workspace > Text File Encoding set it to UTF-8.

See also:

• Unicode - How to get characters right? - Contains detailed answers to various problems.

Answer 2

Is it possible the string is in tact, but you are attempting to print these characters with a en-us localization?

Answer 3

Okay, so the first problem is you need to find out where the data is being lost.

• Add appropriate logging of the unicode characters (ideally in hex) so you can see whether you can write to SQLite and retrieve the data correctly.
• Hard-code some data so you can see whether it's coming back correctly
• Make sure that anywhere you have a text-to-binary conversion, you specify an appropriate encoding (e.g. UTF-8)

You haven't really said where things are going wrong, but I'd expect that if you sort out the character encoding, the rest should fall into place. Maybe SQLite has problems, but I doubt it...

Answer 4

You need to make sure that the HTML that is sent back to the browser has a charset. You should both send back Content-Type: text/html; charset=UTF-8 as an HTTP response header and include, as the first child element of the head tag:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

Or, if you are using XHTML:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

Though just having the meta tag will often fix the problem.

Also, make sure that your HTML is valid by using the W3C Markup Validation Service.

See also: FAQ: Weird characters and question marks appear instead of accented characters