Why non-ascii chars are displayed as weird symbols?

Question

I have 2 cases here:

My Database contains lots of info which I want to fetch to the page, some of these info are name which contain non-ascii chars like Uwe Rülke

- Old solution which works well:

I fetch the data from DB and populate the page directly from a VB while loop. In this case all the chars are displaying correctly Uwe Rülke

- New solution which doesn't work properly:

The VB's While loop doesn't throw the data directly to the page, rather in a JS strings (to enhance performance by not calling DB each now and then). but when I use the info stored in the JS variables I got something like this: Uwe R�lke.

In both cases the page's encoding is :

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

Where did I go wrong ?

EDIT

This is the code used to Fetch (from DB) then save to js Strings.

I'm using AJAX LOAD from a page called ISEquery to build a specific request and query it from DB. It is used to either fetch data as an excel file or as plain HTML. At this point the chars are well represented.

Then the magic happens, and the chars get mis-represented, I checked it in the exctractFields function

$("<div></div>").load("ISEquery.asp?isExcel=0&" + info, function(){
                            // Extracting the fields into an array
                            var rows = "";
                            var options = "";
                            $(this).children().each(function(index){
                                var fieldsArray = exctractFields($(this).html());
                                rows += createISELine(fieldsArray);
                                options += createISELine_ComboBox(fieldsArray);
                            });
                        });

Answer 1

The � means that you used a character which can't be represented properly.

Somewhere between the server and the client, you need to encode the string data properly. I don't know how you transfer the data from the server to the client (generate JavaScript, AJAX, GET requests), it's hard to say how to fix this.

But what you need to do: For every step, you must make sure that you know what the current encoding of the data is and what the recipient expects.

For example if you generate inline JavaScript in a HTML page, then the string value must be encoded with the same encoding as the page (iso-8859-1). If you use AJAX, then usually you have to use UTF-8.

Answer 2

Thanks to Aaron Digulla's answer, I followed the string from server to the page and found that it is gets misrepresented after the AJAX LOAD, so I found this answer which resolved my problem. Although I had to use the charset="iso-8859-1" for it to work rather than charset="UTF-8".

So the final answer is:

-Encoding in the HTML page:

<meta http-equiv="Content-Type" content="text/html"; charset="iso-8859-1">

-Encoding the Ajax data:

 $.ajaxSetup({
          'beforeSend' : function(xhr) {
           xhr.overrideMimeType('text/html; charset=iso-8859-1');
        },
    });

And now chars are displayed correctly. Thanx for your help guys..

Answer 3

Javascript default encoding for strings is UTF-16 (16 bits) while iso-8859-1 is 8 bits.

Default Javascript Character Encoding?

I think you can use encodeURI() to convert your special characters to ASCII characters and afterwards you can decode it with decodeURI()

http://www.w3schools.com/jsref/jsref_encodeuri.asp