NUnit - cleanup after test failure

Question

We have some NUnit tests that access the database. When one of them fails it can leave database in inconsistent state - which is not an issue, since we rebuild database for

Answer 1

Another option is to have a special function that will throw your exceptions, that sets a switch in the testfixture that says an exception occured.

public abstract class CleanOnErrorFixture
{
     protected bool threwException = false;

     protected void ThrowException(Exception someException)
     {
         threwException = true;
         throw someException;
     }

     protected bool HasTestFailed()
     {
          if(threwException)
          {
               threwException = false; //So that this is reset after each teardown
               return true;
          }
          return false;
     }
}

Then using your example:

[TestFixture]
public class SomeFixture : CleanOnErrorFixture
{
    [Test]
    public void MyFailTest()
    {
        ThrowException(new InvalidOperationException());
    }

    [Test]
    public void MySuccessTest()
    {
        Assert.That(true, Is.True);
    }

    [TearDown]
    public void CleanUpOnError()
    {
        if (HasLastTestFailed()) CleanUpDatabase();
    }
}

The only issue here is that the Stack trace will lead to the CleanOnErrorFixture

Answer 2

This idea got me interested, so I did a little digging. NUnit doesn't have this ability out of the box, but there is a whole extensibility framework supplied with NUnit. I found this great article about extending NUnit - it was a good starting point. After playing around with it, I came up with the following solution: a method decorated with a custom CleanupOnError attribute will be called if one of the tests in the fixture failed.

Here's how the test looks like:

  [TestFixture]
  public class NUnitAddinTest
  {
    [CleanupOnError]
    public static void CleanupOnError()
    {
      Console.WriteLine("There was an error, cleaning up...");
      // perform cleanup logic
    }

    [Test]
    public void Test1_this_test_passes()
    {
      Console.WriteLine("Hello from Test1");
    }

    [Test]
    public void Test2_this_test_fails()
    {
      throw new Exception("Test2 failed");
    }

    [Test]
    public void Test3_this_test_passes()
    {
      Console.WriteLine("Hello from Test3");
    }
  }

where the attribute is simply:

  [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
  public sealed class CleanupOnErrorAttribute : Attribute
  {
  }

And here is how it's executed from the addin:

public void RunFinished(TestResult result)
{
  if (result.IsFailure)
  {
    if (_CurrentFixture != null)
    {
      MethodInfo[] methods = Reflect.GetMethodsWithAttribute(_CurrentFixture.FixtureType,
                                                             CleanupAttributeFullName, false);
      if (methods == null || methods.Length == 0)
      {
        return;
      }

      Reflect.InvokeMethod(methods[0], _CurrentFixture);
    }
  }
}

But here's the tricky part: the addin must be placed in the addins directory next to the NUnit runner. Mine was placed next to the NUnit runner in TestDriven.NET directory:

C:\Program Files\TestDriven.NET 2.0\NUnit\addins

(I created the addins directory, it wasn't there)

EDIT Another thing is that the cleanup method needs to be static!

I hacked together a simple addin, you can download the source from my SkyDrive. You will have to add references to nunit.framework.dll, nunit.core.dll and nunit.core.interfaces.dll in the appropriate places.

A few notes: The attribute class can be placed anywhere in your code. I didn't want to place it in the same assembly as the addin itself, because it references two Core NUnit assemblies, so I placed it in a different assembly. Just remember to change the line in the CleanAddin.cs, if you decide to put it anywhere else.

Hope that helps.

Answer 3

How does it fail? Is it possible to put it in a try (do test) / catch (fix broken db) / finally block?

Or you could call a private method to fix it when you've checked your fail condition.

Answer 4

Yes, there is. You can use the Teardown attribute which will teardown after each test. You'd want to apply that Database "reset" script that you have and teardown and re-setup before and after each test.

This attribute is used inside a TestFixture to provide a common set of functions that are performed after each test method is run.

Update: Based on the comments and update to the question, I'd say you can use the teardown attribute and use private variables to indicate whether the method contents should fire.

Though, I did also see that you don't want any complex logic or error handling code.

Given that, I'd think that a standard Setup/Teardown would work best for you. It doesn't matter if there is an error and you don't have to have any error handling code.

If you need to special clean up because the next tests depends on successful completion of the current test, I'd suggest to revisit your tests -- they probably shouldn't depend on each other.

Answer 5

Since version 2.5.7, NUnit allows Teardown to detect if last test failed. A new TestContext class allows tests to access information about themselves including the TestStauts.

For more details, please refer to http://nunit.org/?p=releaseNotes&r=2.5.7

[TearDown]
public void TearDown()
{
    if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
    {
        PerformCleanUpFromTest();
    }
}