Retaining special character while reading from html java?

问题

i am trying to read html source file which contains German characters like ä ö ü ß €

Reading using JSOUP

citAttr.nextElementSibling().text()

Encoding the string with

unicodeEscaper.translate(citAttr.nextElementSibling().text())

org.apache.commons.lang3.text.translate.UnicodeEscaper

Issue is after reading the charecters turns into �

But where as reading CSV with Encoded type UTF-8 with above unicodeEscaper saving & retriving the charecters works fine.

unicodeEscaper.translate(record.get(headerPosition.get(0)))

Whats the issue with reading from html ?? did try with StringUtilEscaper methods still the charecters turns into �

private String getText(Part p) throws MessagingException, IOException {
    if (p.isMimeType("text/*")) {
        String s = (String) p.getContent();
        textIsHtml = p.isMimeType("text/html");
        return s;
    }

This is how i am reading email which have html content!

回答1:

I just answered a similar question today... I guess I can just type what I know about extended character sets (foreign-language characters), since that's one of the major facets of the software I write.

Java's internal String's all use 16-bit chars (The primitive type char is a 16-bit primitive value. The name UTF-8 is a little misleading since it is used to represent the 16-bit "Unicode Space" (using two 8-bit numbers). This means that Java (and Java String's) have no problems representing the entire Unicode foreign-language alphabet ranges.
JSoup, and just about any HTML tool written in Java, when asking for website pages to download, will return 16-bit characters - as Java String's - just fine, without any problems! If there are problems viewing these ranges, it is likely not the download process, nor a JSoup or HttpUrlConnection setting. When you save a web-page to a String in Java, you haven't lost those characters, you essentially get UTF-8 "for free."
HOWEVER: Whenever a programmer attempts to save a UTF-8 String to a '.txt' File or a '.html' File, if you then go on to view that content (that file) in a web-browser, all you might see is that annoying question mark: �. This is because you need to make sure to let your web-browser know that the '.html' File that you have saved using Java - is not intended to be interpreted using the (much older, much shorter) 8-bit ASCII Range.

If you view an '.html' File in any web-browser, or upload that file to Google Cloud Platform (or some hosting site), you must do one of two things:

Include the <META> Tag mentioned in the comments: <meta charset="UTF-8"> in the HTML Page's <HEAD> ... </HEAD> section.

Or provide the setting in whatever hosting platform you have to identify the file as 'text/html, charset=UTF-8'. In Google Cloud Platform Storage Buckets there is a popup menu to assign this setting to any file.

来源：https://stackoverflow.com/questions/64034234/retaining-special-character-while-reading-from-html-java

标签

java

html

unicode

jsoup