问题
Does anyone know if it is possible to read from a socket directly into a location in a memory-mapped .NET file? This happens to be a non-persistent memory-mapped file (purely in memory, no associated disk file), if that helps.
Context: I'm trying to implement minimal-copying code in a situation where I have large memory-mapped objects to move from node to node in a distributed system. I'm using UDP via the standard socket library, so I had hoped to be able to read, say, 64KB off a socket (I'm on an Infiniband network, so the MTU can be quite large) into a location at some offset into the memory-mapped region.
So far everything I'm finding seems to involve doing a copy operation -- reading my data first, then copying it into the memory-mapped file. So that's what I would like to avoid: that copy operation. In fact I have the same issue for sending: I would like to send directly from the memory-mapped file.
While copying may seem minor, the objects I'm working with are huge (a cloud scenario): massive numbers of multi-gigabyte memory-mapped objects, which I would like to treat as byte arrays. So those extra copying operations could be expensive for me. In fact my real goal is to use Infiniband verbs and do direct DMA transfer from a memory-mapped region on machine A to a memory-mapped region on machine B, bypassing UDP entirely.
Any pointers would be VERY appreciated!
(More details: These are applications on big clusters, 64-bit machines, and while my code is in .NET written in C#, the applications creating these memory-mapped objects are mostly in C++ or other languages -- think of them as Hadoop or Mapreduce tasks, for example, and the files as huge images, or concatenated web pages, stuff like that. So those applications produce these files -- maybe output from a Map step -- and now they need to be "shuffled" to the right places. This is the specific thing I'm trying to do... ideally with my code still living in .NET/C#, simply because I like C# in .NET. I cross-compile with Mono for Linux... which is actually what they run on these clusters)
回答1:
Clearly Keith deserves credit for pointing me in the right direction, but to summarize the answer: what would work best for me is to write a little module in C or C++, compile it to create a DLL that would be part of my assembly, and load it (this can be done dynamically or statically; latter is easier of course). I can then call from C# into C and my C code can just do the raw system calls using true addresses. Since memory-mapped files (even the ones with no persistent storage) don't move around due to garbage collection or consolidation, this is an unusually easy case. In fact the C code doesn't even need to learn the address from the C# code -- it can just "remap" the same file, although passing the address should be easy enough. Moreover, this can be just as portable as my original C# code, when you come down to it.
In contrast doing this without a helper procedure or two would be a real pain.
Keith, if you want to repost this under your name, I'll delete my summary and "vote" you to the top (you do deserve the points...)
来源：https://stackoverflow.com/questions/17732844/socket-i-o-directly-into-a-net-memory-mapped-file