How do I concisely implement multiple similar unit tests in the Python unittest framework?

Question

I\'m implementing unit tests for a family of functions that all share a number of invariants. For example, calling the function with two matrices produce a matrix of known s

Answer 1

I see that this question is old. I'm not sure about back then, but today maybe you could use some "data-driven testing" packages:

• https://github.com/wolever/nose-parameterized
• http://ddt.readthedocs.org/en/latest/example.html

Answer 2

The above metaclass code has trouble with nose because nose's wantMethod in its selector.py is looking at a given test method's __name__, not the attribute dict key.

To use a metaclass defined test method with nose, the method name and dictionary key must be the same, and prefixed to be detected by nose (ie with 'test_').

# test class that uses a metaclass
class TCType(type):
    def __new__(cls, name, bases, dct):
        def generate_test_method():
            def test_method(self):
                pass
            return test_method

        dct['test_method'] = generate_test_method()
        return type.__new__(cls, name, bases, dct)

class TestMetaclassed(object):
    __metaclass__ = TCType

    def test_one(self):
        pass
    def test_two(self):
        pass

Answer 3

I've read the above metaclass example, and I liked it, but it was missing two things:

1. How to drive it with a data structure?
2. How to make sure that the test function is written correctly?

I wrote this more complete example, which is data-driven, and in which the test function is itself unit-tested.

import unittest

TEST_DATA = (
    (0, 1),
    (1, 2),
    (2, 3),
    (3, 5), # This intentionally written to fail
)   


class Foo(object):

  def f(self, n):
    return n + 1


class FooTestBase(object):
  """Base class, defines a function which performs assertions.

  It defines a value-driven check, which is written as a typical function, and
  can be tested.
  """

  def setUp(self):
    self.obj = Foo()

  def value_driven_test(self, number, expected):
    self.assertEquals(expected, self.obj.f(number))


class FooTestBaseTest(unittest.TestCase):
  """FooTestBase has a potentially complicated, data-driven function.

  It needs to be tested.
  """
  class FooTestExample(FooTestBase, unittest.TestCase):
    def runTest(self):
      return self.value_driven_test

  def test_value_driven_test_pass(self):
    test_base = self.FooTestExample()
    test_base.setUp()
    test_base.value_driven_test(1, 2)

  def test_value_driven_test_fail(self):
    test_base = self.FooTestExample()
    test_base.setUp()
    self.assertRaises(
        AssertionError,
        test_base.value_driven_test, 1, 3)


class DynamicTestMethodGenerator(type):
  """Class responsible for generating dynamic test functions.

  It only wraps parameters for specific calls of value_driven_test.  It could
  be called a form of currying.
  """

  def __new__(cls, name, bases, dct):
    def generate_test_method(number, expected):
      def test_method(self):
        self.value_driven_test(number, expected)
      return test_method
    for number, expected in TEST_DATA:
      method_name = "testNumbers_%s_and_%s" % (number, expected)
      dct[method_name] = generate_test_method(number, expected)
    return type.__new__(cls, name, bases, dct)


class FooUnitTest(FooTestBase, unittest.TestCase):
  """Combines generated and hand-written functions."""

  __metaclass__ = DynamicTestMethodGenerator


if __name__ == '__main__':
  unittest.main()

When running the above example, if there's a bug in the code (or wrong test data), the error message will contain function name, which should help in debugging.

.....F
======================================================================
FAIL: testNumbers_3_and_5 (__main__.FooUnitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "dyn_unittest.py", line 65, in test_method
    self.value_driven_test(number, expected)
  File "dyn_unittest.py", line 30, in value_driven_test
    self.assertEquals(expected, self.obj.f(number))
AssertionError: 5 != 4

----------------------------------------------------------------------
Ran 6 tests in 0.002s

FAILED (failures=1)