Invalid XML Character During Unmarshall

Question

I am marshalling objects to XML file using encoding "UTF-8". It generates file successfully. But when I try to unmarshal it back, there is an error:

An invalid XML character (Unicode: 0x{2}) was found in the value of attribute "{1}" and element is "0"

The character is 0x1A or \u001a, which is valid in UTF-8 but illegal in XML. Marshaller in JAXB allows writing this character into XML file, but Unmarshaller cannot parse it back. I tried to use another encoding (UTF-16, ASCII, etc) but still error.

The common solution is to remove/replace this invalid character before XML parsing. But if we need this character back, how to get the original character after unmarshalling?

---

While looking for this solution, I want to replace the invalid characters with a substitute character (for example dot = ".") before unmarshalling.

I have created this class:

public class InvalidXMLCharacterFilterReader extends FilterReader {

    public static final char substitute = '.'; 

    public InvalidXMLCharacterFilterReader(Reader in) {
        super(in);
    }

    @Override
    public int read(char[] cbuf, int off, int len) throws IOException {

        int read = super.read(cbuf, off, len);

        if (read == -1)
            return -1;

        for (int readPos = off; readPos < off + read; readPos++) {
            if(!isValid(cbuf[readPos])) {
                   cbuf[readPos] = substitute;
            }
        }

        return readPos - off + 1; 
    }

    public boolean isValid(char c) {
        if((c == 0x9)
                || (c == 0xA) 
                || (c == 0xD) 
                || ((c >= 0x20) && (c <= 0xD7FF)) 
                || ((c >= 0xE000) && (c <= 0xFFFD)) 
                || ((c >= 0x10000) && (c <= 0x10FFFF)))
        {
            return true;
        } else
            return false;
    }
 }

Then this is how I read and unmarshall the file:

FileReader fileReader = new FileReader(this.getFile());
Reader reader = new InvalidXMLCharacterFilterReader(fileReader);
Object o = (Object)um.unmarshal(reader);

Somehow the reader does not replace invalid characters with the character I want. It results a wrong XML data which can't be unmarshalled. Is there something wrong with my InvalidXMLCharacterFilterReader class?

Answer 1

I think the main problem is about escaping illegal characters during marshalling. Something similar was mentioned here, you could try it out.

It offers to change encoding to Unicode marshaller.setProperty("jaxb.encoding", "Unicode");

Answer 2

The Unicode character U+001A is illegal in XML 1.0:

The encoding used to represent it does not matter in this case, it's simply not allowed in XML content.

XML 1.1 allows some of the restricted characters (including U+001A) to be included, but they must be present as numeric character references ()

Wikipedia has a nice summary of the situation.