Deserialization in F# vs. C#

问题

I have the following json:

{
  "data": [
    {
      "timestamp": "2019-11-07T00:23:52.095Z",
      "symbol": "XBTUSD",
      "side": "Buy",
      "size": 1,
      "price": 9356.5,
      "tickDirection": "PlusTick",
      "trdMatchID": "01476235-ad89-1777-9067-8ce6d0e29992",
      "grossValue": 10688,
      "homeNotional": 0.00010688,
      "foreignNotional": 1
    }
  ]
}

The last 3 fields are optional.

When I deserialize it in C#, I do the following:

public class Trade
{
    public DateTime Timestamp;
    public string Symbol;
    public string Side;
    public long Size;
    public long Price;
    public long? GrossValue;
    public float? HomeNotional;
    public float? ForeignNotional;
}

public class TradeContainer
{
    public Trade[] Data;
}

var j = JsonConvert.DeserializeObject<TradeContainer>(x);

and it's all good.

In F#, I do this:

type Trade =
    {
        Timestamp : DateTime
        Symbol : string
        Side : string
        Size : int64
        Price : int64
        GrossValue : int64 option
        HomeNotional : float option
        ForeignNotional : float option
    }

type TradeContainer =
    {
        Data : Trade[]
    }

let t = JsonConvert.DeserializeObject<TradeContainer>(x)

but it will fail. However, if I remove the option keyword, in the Trade type, then it deserializes properly.

The error I get is:

Newtonsoft.Json.JsonSerializationException: Unexpected property 'homeNotional' found when reading union. Path 'data[0].homeNotional', line 1, position 233. at at Newtonsoft.Json.Converters.DiscriminatedUnionConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)

Why the difference? but also how can I achieve this since I need to plan that some values may not be present?

回答1:

The problem is in JSON.NET's implementation of DU serialization. It's not idiomatic and essentially dumps the case and fields :

type Shape =
    | Rectangle of width : float * length : float
    | Circle of radius : float
    | Empty

[<EntryPoint>]
let main argv = 

    let shape1 = Rectangle(1.3, 10.0)

    let json = JsonConvert.SerializeObject(shape1)
    // {
    //   "Case": "Rectangle",
    //   "Fields": [
    //     1.3,
    //     10.0
    //   ]
    // }

The deserializer expects the same structure.

Isaac Abraham created an idiomatic custom converter that should be used instead :

let settings = new JsonSerializerSettings()
settings.Converters.Add(IdiomaticDuConverter())

let t = JsonConvert.DeserializeObject<TradeContainer>(json,settings)

IdiomaticDuConverter's code is :

namespace Newtonsoft.Json.Converters

open Microsoft.FSharp.Reflection
open Newtonsoft.Json
open System

type IdiomaticDuConverter() = 
    inherit JsonConverter()

    [<Literal>]
    let discriminator = "__Case"
    let primitives = Set [ JsonToken.Boolean; JsonToken.Date; JsonToken.Float; JsonToken.Integer; JsonToken.Null; JsonToken.String ]

    let writeValue (value:obj) (serializer:JsonSerializer, writer : JsonWriter) =
        if value.GetType().IsPrimitive then writer.WriteValue value
        else serializer.Serialize(writer, value)

    let writeProperties (fields : obj array) (serializer:JsonSerializer, writer : JsonWriter) = 
        fields |> Array.iteri (fun index value -> 
                      writer.WritePropertyName(sprintf "Item%d" index)
                      (serializer, writer) |> writeValue value)

    let writeDiscriminator (name : string) (writer : JsonWriter) = 
        writer.WritePropertyName discriminator
        writer.WriteValue name

    override __.WriteJson(writer, value, serializer) = 
        let unionCases = FSharpType.GetUnionCases(value.GetType())
        let unionType = value.GetType()
        let case, fields = FSharpValue.GetUnionFields(value, unionType)
        let allCasesHaveValues = unionCases |> Seq.forall (fun c -> c.GetFields() |> Seq.length > 0)

        match unionCases.Length, fields, allCasesHaveValues with
        | 2, [||], false -> writer.WriteNull()
        | 1, [| singleValue |], _
        | 2, [| singleValue |], false -> (serializer, writer) |> writeValue singleValue
        | 1, fields, _
        | 2, fields, false -> 
            writer.WriteStartObject()
            (serializer, writer) |> writeProperties fields
            writer.WriteEndObject()
        | _ -> 
            writer.WriteStartObject()
            writer |> writeDiscriminator case.Name
            (serializer, writer) |> writeProperties fields
            writer.WriteEndObject()

    override __.ReadJson(reader, destinationType, _, _) = 
        let parts = 
            if reader.TokenType <> JsonToken.StartObject then [| (JsonToken.Undefined, obj()), (reader.TokenType, reader.Value) |]
            else 
                seq { 
                    yield! reader |> Seq.unfold (fun reader -> 
                                         if reader.Read() then Some((reader.TokenType, reader.Value), reader)
                                         else None)
                }
                |> Seq.takeWhile(fun (token, _) -> token <> JsonToken.EndObject)
                |> Seq.pairwise
                |> Seq.mapi (fun id value -> id, value)
                |> Seq.filter (fun (id, _) -> id % 2 = 0)
                |> Seq.map snd
                |> Seq.toArray

        let values = 
            parts
            |> Seq.filter (fun ((_, keyValue), _) -> keyValue <> (discriminator :> obj))
            |> Seq.map snd
            |> Seq.filter (fun (valueToken, _) -> primitives.Contains valueToken)
            |> Seq.map snd
            |> Seq.toArray

        let case = 
            let unionCases = FSharpType.GetUnionCases(destinationType)
            let unionCase =
                parts
                |> Seq.tryFind (fun ((_,keyValue), _) -> keyValue = (discriminator :> obj))
                |> Option.map (snd >> snd)
            match unionCase with
            | Some case -> unionCases |> Array.find (fun f -> f.Name :> obj = case)
            | None ->
                // implied union case
                match values with
                | [| null |] -> unionCases |> Array.find(fun c -> c.GetFields().Length = 0)
                | _ -> unionCases |> Array.find(fun c -> c.GetFields().Length > 0)

        let values = 
            case.GetFields()
            |> Seq.zip values
            |> Seq.map (fun (value, propertyInfo) -> Convert.ChangeType(value, propertyInfo.PropertyType))
            |> Seq.toArray

        FSharpValue.MakeUnion(case, values)

    override __.CanConvert(objectType) =
        FSharpType.IsUnion objectType &&
        not (objectType.IsGenericType &&
             typedefof<list<_>> = objectType.GetGenericTypeDefinition())

来源：https://stackoverflow.com/questions/58748946/deserialization-in-f-vs-c-sharp