Deadlocks causing 'Server failed to resume the transaction' with NHibernate and distributed transactions

Question

We are having an issue when using NHibernate with distributed transactions.

Consider the following snippet:

//
// There is already an ambient distributed         


        

Answer 1

We've finally narrowed this down to a cause.

When opening a session, if there is an ambient distributed transaction, NHibernate attaches an event handler to the Transaction.TransactionCompleted, which closes the session when the distributed transaction is completed. This appears to be subject to a race condition wherein the connection may be closed and returned to the pool before the deadlock error propagates across, leaving the connection in an unusable state.

The following code will reproduce the error for us occasionally, even without any load on the server. If there is extreme load on the server, it becomes more consistent.

using(var scope = new TransactionScope()) {
    //
    // Force promotion to distributed transaction
    //
    TransactionInterop.GetTransmitterPropagationToken(Transaction.Current);

    var connection = new SqlConnection(_connectionString);
    connection.Open();

    //
    // Close the connection once the distributed transaction is
    // completed.
    //
    Transaction.Current.TransactionCompleted += 
        (sender, e) => connection.Close();

    using(connection.BeginTransaction())
        //
        // Deadlocks but sometimes does not raise exception
        //
        ForceDeadlockOnConnection(connection);

    scope.Complete();
}

//
// Subsequent attempts to open a connection with the same
// connection string will fail
//

We have not settled on a solution, but the following things will eliminate the problem (while possibly having other consequences):

• Turning off connection pooling
• Using NHibernate's AdoNetTransactionFactory instead of AdoNetWithDistributedTransactionFactory
• Adding error handling that calls SqlConnection.ClearPool() when the "server failed to resume the transaction" error occurs

According to Microsoft (https://connect.microsoft.com/VisualStudio/feedback/details/722659/), the SqlConnection class is not thread-safe, and that includes closing the connection on a separate thread. Based on this response we have filed a bug report for NHibernate (http://nhibernate.jira.com/browse/NH-3023).