Sharing data between AppDomains

Question

I have a process that can have multiple AppDomains. Each AppDomain collect some statistics. After a specified time, I want to accumulate these statistic and save them into a file.

One way to do this is Remoting, which I want to avoid.

The only other technique I have in mind is to save each AppDomain's data in a file, and after a specific time, one of the AppDomain collects all data and accumulate them.

But it would be ideal if this all could be done in-memory, without the cost of serializing the information to pass between AppDomains. Anyone have any ideas?

Answer 1

The only way to avoid serialisation is to represent your data using objects which derive from MarshalByRefObject, but in that case you will still have the cost of marshalling across the AppDomain boundaries. This may also involve the refactoring/re-writing of much of your code.

Assuming marshalling by reference is not an option, you will have to serialise at some point. It simply cannot be avoided. One way to do this is as Neil Barnwell suggests, with a database, another would be with a local file as you suggest yourself.

Another way which may or may not feasible depending on your delivery timeline and/or .NET 4.0 adoption, would be to use a memory mapped file, see .Net Framework 4.0: Using memory mapped files.

Answer 2

It is possible to share data between AppDomains without the costs of Marshalling. But it is a rather hacky way. You can create a source data object which is shared by reference between all AppDomains. This way you get all data into one shared object without the costs of Marshalling. Sounds too easy to be true?

The first thing is to know how share data between AppDomains without Marshalling. For this you get the object address of your data source object via Marshal.UnsafeAddrOfPinnedArrayElement. Then you pass this IntPtr to all AppDomains which are interested in this. In the target AppDomain you need to cast this IntPtr back to an object reference which can be done JIT::CastAny which is done if you return an object from a method and push the pointer of it onto the stack.

Viola you have shared an object as plain pointer between AppDomains and you get InvalidCastExceptions. The problem is that you must set for all your AppDomains LoaderOptimization.MultiDomain to ensure that the assembly that defines the shared data type is loaded as AppDomain neutral type which has the same Method Table pointer between all AppDomains.

You can find an example application that does exactly this as part of WMemoryProfiler. See this link for a more detailed explanation and download link to the sample code.

The basic code is

[LoaderOptimization(LoaderOptimization.MultiDomain)]
static public void Main(string[] args)
{

    // To load our assembly appdomain neutral we need to use MultiDomain on our hosting and child domain
    // If not we would get different Method tables for the same types which would result in InvalidCastExceptions
    // for the same type.
    var other = AppDomain.CreateDomain("Test"+i.ToString(), AppDomain.CurrentDomain.Evidence, new AppDomainSetup
        {
            LoaderOptimization = LoaderOptimization.MultiDomain,
        });

    // Create gate object in other appdomain
    DomainGate gate = (DomainGate)other.CreateInstanceAndUnwrap(Assembly.GetExecutingAssembly().FullName, typeof(DomainGate).FullName);

    // now lets create some data
    CrossDomainData data = new CrossDomainData();
    data.Input = Enumerable.Range(0, 10).ToList();

    // process it in other AppDomain
    DomainGate.Send(gate, data);

    // Display result calculated in other AppDomain
    Console.WriteLine("Calculation in other AppDomain got: {0}", data.Aggregate);
    }
}

Answer 3

I tend to say just use remoting. Writing the data to a file requires serialization, too. Serialization seems to be almost unavoidable what ever technology you use. You have to transfer data from one application domain to another using some channel and you will have to serialize the data in order to get it through the channel.

The only way to avoid serialization seems to be using shared memory so that both application domains can access the data without ever going through a channel. Even deep cloning the data from one application domain's memory into the other's memory is at its core nothing more then a binary serialization (where the result is not necessarily stored in consecutive memory locations).

Answer 4

I do appreciate you want to keep this in-memory, but my first suggestion would be to write the data to a database and query from there. Remoting is still a remote call, which is where much of the "cost" of using a database server comes from, and you'd have to build in transaction-handling to make sure you don't lose data. If you write to a SQL Server database you have transaction support ready and waiting for you, and it's fast-fast-fast for queries.