Deserializing base64 encoded byte arrays return different results in 2 JSON.net library versions

Question

  JObject j = JObject.Parse(\"{\'responseArray\':\'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA\'}\");

  byte[] r = j[\"responseArray\"].ToObject(JsonSerializer.C         


        

Answer 1

Update

This is fixed in Json.NET 7.0 Release 1.

Original Answer

This is a Json.NET issue. The problem is that your string AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA just happens to be successfully parsable as a GUID, in JsonReader.ReadAsBytesInternal():

            if (s.Length == 0)
            {
                data = new byte[0];
            }
            else if (ConvertUtils.TryConvertGuid(s, out g))
            {
                data = g.ToByteArray();
            }
            else
            {
                data = Convert.FromBase64String(s);
            }

At this traceback:

Newtonsoft.Json.JsonReader.ReadAsBytesInternal() Line 517   C#
Newtonsoft.Json.Linq.JTokenReader.ReadAsBytes() Line 74 + 0x9 bytes C#
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadForType(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, Newtonsoft.Json.Serialization.JsonContract contract = {Newtonsoft.Json.Serialization.JsonPrimitiveContract}, bool hasConverter = false) Line 1853 + 0x8 bytes   C#
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}, bool checkAdditionalContent = false) Line 144 + 0x2f bytes   C#
Newtonsoft.Json.JsonSerializer.DeserializeInternal(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 710 + 0x52 bytes    C#
Newtonsoft.Json.JsonSerializer.Deserialize(Newtonsoft.Json.JsonReader reader = {Newtonsoft.Json.Linq.JTokenReader}, System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 689 + 0x11 bytes    C#
Newtonsoft.Json.Linq.JToken.ToObject(System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}, Newtonsoft.Json.JsonSerializer jsonSerializer = {Newtonsoft.Json.JsonSerializer}) Line 1837 + 0x11 bytes    C#
Newtonsoft.Json.Linq.JToken.ToObject(System.Type objectType = {Name = "Byte[]" FullName = "System.Byte[]"}) Line 1811 + 0x3c bytes  C#
Newtonsoft.Json.Linq.JToken.ToObject<byte[]>() Line 1698 + 0x39 bytes   C#

As you can see, this short-circuits the base 64 decoding. Ugly bug, and rather bad luck on your part to get that exact string.

To block Json.NET's inappropriate GUID recognition you can create your own global JsonConverter for byte arrays. When you do, Json.NET will pass you the raw string without the "helpful" GUID pattern match:

public class ByteConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(byte[]);
    }

    public override bool CanWrite { get { return false; } } // Use the default implementation for serialization, which is not broken.

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var value = (string)JToken.Load(reader);
        if (value == null)
            return null;
        if (value.Length == 0)
            return new byte[0];
        return Convert.FromBase64String(value);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

And then, to set it globally:

        JsonConvert.DefaultSettings = () => new JsonSerializerSettings
        {
            Converters = new List<JsonConverter> { new ByteConverter() }
        };

This workaround restores the correct deserialization for byte arrays.

Answer 2

The problem is that JObject has already parsed the AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA as a string (j["responseArray"].Type == JTokenType.String), so the j["responseArray"].ToObject<byte[]> isn't doing a de-base64.

You have to parse it directly to a byte[], like:

public class MyObject
{
    public byte[] responseArray { get; set; }
}

MyObject cl = JsonConvert.DeserializeObject<MyObject>("{'responseArray':'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}");

Clearly you can do the Convert.FromBase64String "manually":

JObject j = JObject.Parse("{'responseArray':'AAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}");
byte[] r = Convert.FromBase64String((string)j["responseArray"]);