Sending explicit zeroes in protobuf3

问题

In Protobuf3 zero is the default value for numeric types, and so they are filtered out when serialized.

I have an application where I need to send a value only when it has changed. For example, x was 1, now x is 0, send this value.

It isn't possible to send only the delta, eg -1, because some of these values are floats or doubles, and we do not want to accrue errors.

There are over 200 different variables in some classes I need to serialize, so solutions like "add a boolean to flag which fields have changed" are possible but not fun. Other suggestions that have a large amount of per-field work or processing are undesirable too.

Is there a simple mechanism to tell protobuf3 to explicitly keep a value even though it is the default value?

Possible solutions:

Send the entire class each time. The main downside here is some fields may have a lot of data.
Use a boolean "has changed" in the schema to indicate if a variable has changed, even if it is 0
Use a magic value. Terrible idea, but possible. Not going to do this.

回答1:

If you need to distinguish 0 and null then you can use proto3 wrapper types: https://developers.google.com/protocol-buffers/docs/reference/csharp-generated#wrapper_types There are special wrapper types for such case: StringWrapper, Int32Wrapper and etc. All of the wrapper types that correspond to C# value types (Int32Wrapper, DoubleWrapper, BoolWrapper etc) are mapped to Nullable<T> where T is the corresponding non-nullable type.

回答2:

Since you tagged protobuf-net, you can do this at the field level:

[ProtoMember(..., IsRequired = true)]
// your field

or globally (here I'm assuming you are using the default model, which is a pretty safe assumption usually):

RuntimeTypeModel.Default.ImplicitZeroDefault = false;

and you're done;

Note: if you're interested in deltas, you can also do this conditionally - for a property Foo, you can add:

private bool ShouldSerializeFoo() { /* your rules here */ }

(this is a name-based pattern used by many serializers and other tools; in some scenarios, it needs to be public, but protobuf-net is usually happy with it non-public)

As a non-trivial example of an object that tracks delta state internally:

using ProtoBuf;
using System;
using System.Collections.Generic;
using System.IO;

static class P
{
    static void Main()
    {
        var obj = new MyType
        {
            Foo = 42,
            Bar = "abc",
            Blap = DateTime.Now
        };
        ShowPayloadSize("original", obj);
        obj.MarkClean();
        ShowPayloadSize("clean", obj);

        obj.Foo = 42;
        obj.Bar = "abc";
        ShowPayloadSize("set property to same", obj);

        obj.Foo = 45;
        obj.Bar = "new value";
        ShowPayloadSize("set property to different", obj);

        obj.MarkClean();
        ShowPayloadSize("clean again", obj);
    }
    static void ShowPayloadSize<T>(string caption, T obj)
    {
        using var ms = new MemoryStream();
        Serializer.Serialize(ms, obj);
        Console.WriteLine($"{caption}: {ms.Length} bytes");
    }
}


[ProtoContract]
public class MyType
{


    private int _dirty = -1; // treat everything as dirty by default
    public void MarkClean() => _dirty = 0;
    public bool IsDirty => _dirty != 0;

    private bool ShouldSerialize(int flag) => (_dirty & flag) != 0;

    private void Set<T>(ref T field, T value, int flag)
    {
        if (!EqualityComparer<T>.Default.Equals(field, value))
        {
            field = value;
            _dirty |= flag;
        }
    }

    [ProtoMember(1)]
    public int Foo
    {
        get => _foo;
        set => Set(ref _foo, value, 1 << 0);
    }
    public bool ShouldSerializeFoo() => ShouldSerialize(1 << 0);
    private int _foo;

    [ProtoMember(2)]
    public string Bar
    {
        get => _bar;
        set => Set(ref _bar, value, 1 << 1);
    }
    public bool ShouldSerializeBar() => ShouldSerialize(1 << 1);
    private string _bar;

    [ProtoMember(3)]
    public DateTime Blap
    {
        get => _blap;
        set => Set(ref _blap, value, 1 << 2);
    }
    public bool ShouldSerializeBlap() => ShouldSerialize(1 << 2);
    private DateTime _blap;
}

来源：https://stackoverflow.com/questions/61931285/sending-explicit-zeroes-in-protobuf3

标签

protocol-buffers

protobuf-net