1. 技术文章
  2. Transferring large payloads of data (Serialized Objects) using wsHttp in WCF with message security

Transferring large payloads of data (Serialized Objects) using wsHttp in WCF with message security

由 自作多情 提交于 2019-11-28 04:34:59

protobuf-net generally has a significant space-saving (as in: order-of-magnitude) on most data, and can attach to WCF. Unfortunately at the moment it doesn't support full graphs, only trees. However, I have plans there that I simply haven't had time to implement. I can't promise anything, but I could try to bump that work a bit sooner.

Otherwise; there may be ways to tweak your existing code to work with a tree instead of a graph.

If you still want to use Message Security, I would recommend you to use MTOM to optimize the network bandwidth that needs be used to transfer the messages, and also the chunking channel for using smaller memory buffers when security is applied. Otherwise, WCF will try to load the whole message in memory to apply security, and therefore you will get the Insufficient memory exception.

Bonshington

i used to implement kinda passing big text to/from wcf. my trig is convert it to stream and use GZipStream to compress it then send it as byte[], luckily its never exceed 10 MB.

In your case i recommend do fragmentation. Convert Serialized object to byte[] and then merg it and decompress

psudo

int transferSize = 5000000; // 5MB
byte[] compressed = ...;
var mem = new System.IO.MemoryStream(compressed);

for(int i = 0; i < compressed .length; i+= transferSize )
{
    byte[] buffer = new byte[transferSize];
    mem.Read(buffer, i, compressed);
    mem.Flush();
    sendFragmentToWCF(buffer);
}

edit 08 Dec 2010

ok based on my understanding, the situation is client is download some large serialize object throught WCF. i didn't particularly test on this solution, but guess it should work. The key is save serialized object to file and use Response transmit that file.

[WebMethod]
public void GetSerializedObject()
{
    string path = @"X:\temp.tmp";

    var serializer = new  System.Runtime.Serialization.NetDataContractSerializer();
    var file = new System.IO.FileStream(path, System.IO.FileMode.CreateNew);

    try
    {
        serializer.Serialize(file, ...);
        this.Context.Response.TransmitFile(path);
        this.Context.Response.Flush();
    }
    finally
    {
        file.Flush();
        file.Close();
        file.Dispose();
        System.IO.File.Delete(path);
    }
}

WCF shoud do file streaming automatically and u dont ahve to worry about serialized object size since we use file transmit. Dont forget to the config response limit.

Kit

Some lighter, but not guaranteed solutions, would be to

  • use the DataContractSerializer instead since you own both sides. This does not require embedded type information, which is significantly large.
  • use [DataMember(EmitDefaultValue = false)] which is discussed in a question I asked - again because you own both sides; doing so will cut down some on the message size (how much depends of course on how many fields in the graph are defaults).
  • use [DataContract(IsReference=true)], especially if you have many repeated value objects or reference data
  • use some sort of throttling on the server to reduce memory pressure of simultaneous results

These are, of course trade-offs, for example with readability.

Since nobody has put it out there, perhaps using WebSockets or long polling techniques may be able to solve this issue. I've only looked into these solutions briefly and haven't developed a solution around them but I wanted to propose these concepts for the record and I'll expand upon my answer at a later point if time permits.

The underlying idea would be to achieve something similar to how the idea of how the ChunkingChannel example works but while not requiring a full duplex channel which typically breaks the port 80 web based request/response model that is desirable to avoid having to make firewall and other related configurations for clients.

Other related material:

Update: After researching more on this it appears that by using WebSockets, which is known known as NetHttpBinding, that I would inherently not be solving the original request which is to use wsHttp in WCF with message security. I'm going to keep my answer here however as information for others who may be looking into an alternative.

标签
wcf
large-data-volumes
wshttpbinding
chunking
netdatacontractserializer
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!