问题
I'm having some issues trying to serialize/deserialize a complex object graph using protobuf-net.
I'm working on a legacy application and we're using .Net Remoting to connect a GUI client to a C# service. We are seeing poor performance with overseas users due to the serialized size of our object graphs using the default BinaryFormatter, which is exacerbated by the limited bandwidth in-between the client and server (1Mbit/s).
As a quick win, I thought I'd put together a proof of concept to see if there were any performance gains to be had by using protobuf-net instead, by implementing ISerializable. As I was testing I ran into an issue whereby object references weren't being maintained.
I've put together an example which repros the issue. I'm expecting that the object in the Dictionary (Items[1]) and the object B.A will be the same as I've specified AsReference=true in the ProtoMember attribute.
Using protobuf-net 2.0.0.619, I'm seeing an exception thrown when deserializing (A reference-tracked object changed reference during deserialization).
If this isn't a supported scenario the please let me know.
Test
[Test]
public void AreObjectReferencesSameAfterDeserialization()
{
A a = new A();
B b = new B();
b.A = a;
b.Items.Add(1, a);
Assert.AreSame(a, b.A);
Assert.AreSame(b.A, b.Items[1]);
B deserializedB;
using (var stream = new MemoryStream())
{
Serializer.Serialize(stream, b);
stream.Seek(0, SeekOrigin.Begin);
deserializedB = Serializer.Deserialize<B>(stream);
}
Assert.AreSame(deserializedB.A, deserializedB.Items[1]);
}
Class definitions
[Serializable]
[ProtoContract]
public class A
{
}
[Serializable]
[ProtoContract]
public class B
{
[ProtoMember(1, AsReference = true)]
public A A { get; set; }
[ProtoMember(2, AsReference = true)]
public Dictionary<int, A> Items { get; set; }
public B()
{
Items = new Dictionary<int, A>();
}
}
回答1:
Edit: this should work from the next build onwards simply by marking the type's AsReferenceDefault:
[ProtoContract(AsReferenceDefault=true)]
public class A
{
// ...
}
At the current time this is sort of an unsupported scenario - at least, via the attributes it is unsupported; basically, the AsReference=true currently is referring to the KeyValuePair<int,A>, which doesn't really make sense since KeyValuePair<int,A> is a value-type (so this can never be treated as a reference; I've added a better message for that in my local copy).
Because KeyValuePair<int,A> acts (by default) as a tuple, there is currently nowhere to support the AsReference information, but that is a scenario I would like to support better, and I will be investigating this.
There was also a bug that meant that AsReference on tuples (even reference-type tuples) was getting out-of-order, but I've fixed that locally; this was where the "changed" message came from.
In theory, the work for me to do this isn't huge; the fundamentals already work, and oddly enough it came up separately on twitter last night too - I guess "dictionary pointing to an object" is a very common scenario. At a guess, I imagince I'll add some atribute to help describe this situation, but you can actually hack around it at the moment using a couple of different routes:
1: configure KeyValuePair<int,A> manually:
[Test]
public void ExecuteHackedViaFields()
{
// I'm using separate models **only** to keep them clean between tests;
// normally you would use RuntimeTypeModel.Default
var model = TypeModel.Create();
// configure using the fields of KeyValuePair<int,A>
var type = model.Add(typeof(KeyValuePair<int, A>), false);
type.Add(1, "key");
type.AddField(2, "value").AsReference = true;
// or just remove AsReference on Items
model[typeof(B)][2].AsReference = false;
Execute(model);
}
I don't like this much, because it exploits implementation details of KeyValuePair<,> (the private fields), and may not work between .NET versions. I would prefer to replace KeyValuePair<,> on the fly via a surrogate:
[Test]
public void ExecuteHackedViaSurrogate()
{
// I'm using separate models **only** to keep them clean between tests;
// normally you would use RuntimeTypeModel.Default
var model = TypeModel.Create();
// or just remove AsReference on Items
model[typeof(B)][2].AsReference = false;
// this is the evil bit: configure a surrogate for KeyValuePair<int,A>
model[typeof(KeyValuePair<int, A>)].SetSurrogate(typeof(RefPair<int, A>));
Execute(model);
}
[ProtoContract]
public struct RefPair<TKey,TValue> {
[ProtoMember(1)]
public TKey Key {get; private set;}
[ProtoMember(2, AsReference = true)]
public TValue Value {get; private set;}
public RefPair(TKey key, TValue value) : this() {
Key = key;
Value = value;
}
public static implicit operator KeyValuePair<TKey,TValue>
(RefPair<TKey,TValue> val)
{
return new KeyValuePair<TKey,TValue>(val.Key, val.Value);
}
public static implicit operator RefPair<TKey,TValue>
(KeyValuePair<TKey,TValue> val)
{
return new RefPair<TKey,TValue>(val.Key, val.Value);
}
}
This configures something to use instead of KeyValuePair<int,A> (converted via the operators).
In both of these, Execute is just:
private void Execute(TypeModel model)
{
A a = new A();
B b = new B();
b.A = a;
b.Items.Add(1, a);
Assert.AreSame(a, b.A);
Assert.AreSame(b.A, b.Items[1]);
B deserializedB = (B)model.DeepClone(b);
Assert.AreSame(deserializedB.A, deserializedB.Items[1]);
}
I do, however, want to add direct support. The good thing about both of the above is that when I get time to do that, you just have to remove the custom configuration code.
For completeness, if your code is using Serializer.* methods, then rather than create / configure a new model, you should configure the default model:
RuntimeTypeModel.Default.Add(...); // etc
Serializer.* is basically a short-cut to RuntimeTypeModel.Default.*.
Finally: you should not create a new TypeModel per call; that would hurt prerformance. You should create and configure one model instance, and re-use it lots. Or just use the default model.
回答2:
I've setup a small test and found out that the AsReferenceDefault attribute doesn't work quite as expected.
Test class:
[ProtoContract(AsReferenceDefault = true)]
public class TEST
{
[ProtoMember(1018)]
public List<TEST> _Items { get; set; }
[ProtoMember(1001, AsReference = true)]
public TEST Parent;
[ProtoMember(1003)]
public string NameItemType;
public void AddItem(TEST Item)
{
_Items.Add(Item);
Item.Parent = this;
}
public TEST()
{
}
}
Test code:
TEST ci = new TEST(); ci._Items = new List<TEST>(); ci.NameItemType = "ROOT_ITEM";
TEST ci_2 = new TEST(); ci_2._Items = new List<TEST>(); ci_2.NameItemType = "ITEM_02"; ci.AddItem(ci_2);
TEST ci_3 = new TEST(); ci_3._Items = new List<TEST>(); ci_3.NameItemType = "ITEM_03"; ci_2.AddItem(ci_3);
// --> Confirm references.
bool AreEqual = false;
if (ci == ci_2.Parent)
AreEqual = true;
if (ci_2 == ci_3.Parent)
AreEqual = true;
// --> Serialize.
byte[] buf;
using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
{
ProtoBuf.Serializer.Serialize(ms, ci);
buf = ms.ToArray();
}
// --> Deserialize.
using (System.IO.MemoryStream ms = new System.IO.MemoryStream(buf))
{
ci = ProtoBuf.Serializer.Deserialize<TEST>(ms);
}
// --> Confirm references.
ci_2 = ci._Items[0];
ci_3 = ci_2._Items[0];
if (ci == ci_2.Parent)
AreEqual = true;
if (ci_2 == ci_3.Parent) // HERE IS WHERE IT FAILS!
// THEY SHOULD BE EQUAL AFTER DESERIALIZATION!
AreEqual = true;
回答3:
Update for those who might come here with the similar problem: starting from version 2.3.0 there is no need to use any tricks Marc mentioned above. Everything works as the Topic Starter wanted:
[TestClass]
public class UnitTest1
{
[TestMethod]
public void AreObjectReferencesSameAfterDeserialization()
{
A a = new A();
B b = new B();
b.A = a;
b.Items.Add( 1, a );
Assert.AreSame( a, b.A );
Assert.AreSame( b.A, b.Items[ 1 ] );
B deserializedB;
var model = TypeModel.Create();
using( var stream = new MemoryStream() )
{
model.Serialize( stream, b );
stream.Seek( 0, SeekOrigin.Begin );
deserializedB = (B) model.Deserialize( stream, null, typeof(B) );
}
Assert.AreSame( deserializedB.A, deserializedB.Items[ 1 ] );
}
}
[ProtoContract]
public class A
{
}
[ProtoContract]
public class B
{
[ProtoMember( 1, AsReference = true )]
public A A { get; set; }
[ProtoMember( 2, AsReference = true )]
public Dictionary<int, A> Items { get; set; }
public B()
{
Items = new Dictionary<int, A>();
}
}
来源：https://stackoverflow.com/questions/14436606/protobuf-net-object-reference-deserialization-using-dictionary-a-reference-trac