.NET 4.0 code contracts - How will they affect unit testing?

Question

For example this article introduces them.

What is the benefit?

Static analysis seems cool but at the same time it would prevent the ability to pass null as a parameter in unit test. (if you followed the example in the article that is)

While on the topic of unit testing - given how things are now surely there is no point for code contracts if you already practice automated testing?

Update

Having played with Code Contracts I'm a little disappointed. For example, based on the code in the accepted answer:

public double CalculateTotal(Order order)
{
    Contract.Requires(order != null);
    Contract.Ensures(Contract.Result<double>() >= 0);
    return 2.0;
}

For unit testing, you still have to write tests to ensure that null cannot be passed, and the result is greater than or equal to zero if the contracts are business logic. In other words, if I was to remove the first contract, no tests would break, unless I had specifically had a test for this feature. This is based on not using the static analysis built into the better (ultimate etc...) editions of Visual Studio however.

Essentially they all boil down to an alternate way of writing traditional if statements. My experience actually using TDD, with Code Contracts shows why, and how I went about it.

Answer 1

I don't think unit testing and contracts interfere with each other that much, and if anything contracts should help unit testing since it removes the need to add tedious repetitive tests for invalid arguments. Contracts specify the minimum you can expect from the function, whereas unit tests attempt to validate the actual behaviour for a particular set of inputs. Consider this contrived example:

public class Order
{
    public IEnumerable Items { get; }
}

public class OrderCalculator
{
    public double CalculateTotal(Order order)
    {
        Contract.Requires(order != null);
        Contract.Ensures(Contract.Result<double>() >= 0);

        return 2.0;
    }
}

Clearly the code satisfies the contract, but you'd still need unit testing to validate it actually behaves as you'd expect.

Answer 2

What is the benefit?

Let's say that you want to make sure that a method never returns null. Now with unit tests, you have to write a bunch of test cases where you call the method with varying inputs and verify that the output is not null. Trouble is, you can't test all possible inputs.

With code contracts, you just declare that the method never returns null. The static analyzer will then complain if it is not possible to prove that. If it doesn't complain, you know that your assertion is correct for all possible inputs.

Less work, perfect correctness guarantees. What's not to like?

Answer 3

Contracts allow you say what the actual purpose of the code is, as opposed to letting whatever the code does with whatever random arguments are handed it standing as the definition from the point of view of the compiler, or the next reader of the code. This allows significantly better static analysis and code optimization.

For instance, if I declare an integer parameter (using the contract notation) to be in the range of 1 to 10, and I have a local array in my function declared the same size, that is indexed by the parameter, the compiler can tell that there is no possibility of subscript error, thus producing better code.

You can state that null is valid value in a contract.

The purpose of unit testing is to verify dynamically that the code achieves whatever stated purpose it has. Just because you've written a contract for a function, doesn't mean the code does that, or that static analysis can verify the code does that. Unit testing won't go away.

Answer 4

Well it will not interfere with unit-testing in general. But as I saw you mentioned something about TDD.

If I think about it from that perspective I guess it could/may change the procedure from the standard one

• create method (just signature)
• create Unit test -> implement the test
• run the test: let it fail
• implement the method, hack it to the end just to make it working
• run the test: see it pass
• refactor your (possibly messy) method body
• (re-run the test just to see you've not broken anything)

This would be the really hard-full-featured unit-testing procedure. In such a context I guess you could insert code contracts between the 1st and 2nd point like

• create method (just signature)
• insert code contracts for the methods input parameters
• create Unit test -> implement the test
• ...

The advantage I see at the moment is that you can write easier unit tests in the sense that you wouldn't have to check every possible path since some is already taken into account by your defined contracts. It just gives you additional checking, but it wouldn't replace unit testing since there will always be more logic within the code, more path that have to be tested with unit tests as usual.

Edit

Another possibility I didn't consider before would be to add the code contracts in the refactoring part. Basically as additional way of assuring things. But that would somehow be redundant and since people don't like to do redundant stuff...