abstract test case using python unittest

Question

Is it possible to create an abstract TestCase, that will have some test_* methods, but this TestCase won't be called and those methods will only be used in subclasses? I think I am going to have one abstract TestCase in my test suite and it will be subclassed for a few different implementation of a single interface. This is why all test methods are the some, only one, internal method changes. How can I do it in elegant way?

Answer 1

I didn't quite understand what do you plan to do -- the rule of thumb is "not to be smart with tests" - just have them there, plain written.

But to achieve what you want, if you inherit from unittest.TestCase, whenever you call unittest.main() your "abstract" class will be executed - I think this is the situation you want to avoid.

Just do this: Create your "abstract" class inheriting from "object", not from TestCase. And for the actual "concrete" implementations, just use multiple inheritance: inherit from both unittest.TestCase and from your abstract class.

import unittest

class Abstract(object):
    def test_a(self):
        print "Running for class", self.__class__

class Test(Abstract, unittest.TestCase):
    pass

unittest.main()

update: reversed the inheritance order - Abstract first so that its defintions are not overriden by TestCase defaults, as well pointed in the comments bellow.

Answer 2

Multiple inheritance isn't a great option here, chiefly for the two following reasons:

1. None of the methods in TestCase use super() so you'd have to list your class first for methods like setUp() and tearDown() to work.
2. pylint will warn that the base class uses self.assertEquals() etc which aren't defined on self at that point.

Here's the kludge I came up with: turn run() into a no-op for the base class only.

class TestBase( unittest.TestCase ):

  def __init__( self, *args, **kwargs ):
    super( TestBase, self ).__init__( *args, **kwargs )
    self.helper = None
    # Kludge alert: We want this class to carry test cases without being run
    # by the unit test framework, so the `run' method is overridden to do
    # nothing.  But in order for sub-classes to be able to do something when
    # run is invoked, the constructor will rebind `run' from TestCase.
    if self.__class__ != TestBase:
      # Rebind `run' from the parent class.
      self.run = unittest.TestCase.run.__get__( self, self.__class__ )                          
    else:
      self.run = lambda self, *args, **kwargs: None

  def newHelper( self ):
    raise NotImplementedError()

  def setUp( self ):
    print "shared for all subclasses"
    self.helper = self.newHelper()

  def testFoo( self ):
    print "shared for all subclasses"
    # test something with self.helper

class Test1( TestBase ):
  def newHelper( self ):
    return HelperObject1()

class Test2( TestBase ):
  def newHelper( self ):
    return HelperObject2()

Answer 3

There's a very simple way that everyone has missed so far. And unlike several of the answers, it works with all test drivers, rather than failing the minute you switch between them.

Simply use inheritence as usual, then add:

del AbstractTestCase

at the end of the module.

Answer 4

Just to put in my two-cents, although it likely goes against some convention, you could define your abstract test case as a protected member to prevent its execution. I've implemented the following in Django and works as required. See example below.

from django.test import TestCase


class _AbstractTestCase(TestCase):

    """
    Abstract test case - should not be instantiated by the test runner.
    """

    def test_1(self):
        raise NotImplementedError()

    def test_2(self):
        raise NotImplementedError()


class TestCase1(_AbstractTestCase):

    """
    This test case will pass and fail.
    """

    def test_1(self):
        self.assertEqual(1 + 1, 2)


class TestCase2(_AbstractTestCase):

    """
    This test case will pass successfully.
    """

    def test_1(self):
        self.assertEqual(2 + 2, 4)

    def test_2(self):
        self.assertEqual(12 * 12, 144)

Answer 5

If you follow the convention of explicitly listing all test classes in run_unittest (see e.g. the Python test suite for many uses of that convention), then it will be straight-forward to not list a specific class.

If you want to continue using unittest.main, and if you can allow using unittest2 (e.g. from Python 2.7), you can use its load_tests protocol to specify which classes contain test cases). In earlier versions, you will have to subclass TestLoader, and override loadTestsFromModule.

Answer 6

Python unittest library has load_tests protocol, which can be used to achieve exactly what do you want:

# Add this function to module with AbstractTestCase class
def load_tests(loader, tests, _):
    result = []
    for test_case in tests:
        if type(test_case._tests[0]) is AbstractTestCase:
            continue
        result.append(test_case)
    return loader.suiteClass(result)

Answer 7

If you really want to use inheritance instead of mixins, a simple solution is to nest the abstract test in another class.

It avoids issues with test runner discovery and you can still import the abstract test from another module.

import unittest

class AbstractTests(object):
    class AbstractTest(unittest.TestCase)
        def test_a(self):
            print "Running for class", self.__class__

class Test(AbstractTests.AbstractTest):
    pass

Answer 8

Another reason for wanting to do what the OP is doing is to create a highly-parameterized base class which implements much of a set of core tests which need to be reproduced in several environments/scenarios. What I'm describing is essentially creating a parameterized fixture, a la pytest, using unittest.

Assuming you (like me) decide to run away as fast as you can from any multiple-inheritance-based solutions, one might have the following problem with using load_tests() to filter out your base class from the loaded suite:

In the standard TestLoader, load_tests is called after the auto-loading-from-class is done. Because: * this auto-loading-from-class will attempt to construct instances from your base class with the standard signature init(self, name), and * you may want this base class to have a very different ctor signature, or * you may want to skip construction-then-removal of your base class instances for some other reason

.. you may want to completely prevent this auto-loading of test instances from base classes.

EDIT: Vadim's solution in this other thread is a more elegant, concise, and independent way to do this. I have implemented the "nested class trick" and confirmed it works beautifully for the purpose of preventing TestLoader from "finding" your TestCase bases.

I originally had done this by modifying TestLoader.loadTestsFromModule to simply skip any TestCase classes which serve as base classes for any other TestCase classes in the module:

for name in dir(module):
    obj = getattr(module, name)
    # skip TestCase classes:
    # 1. without any test methods defined
    # 2. that are base classes
    #    (we don't allow instantiating TestCase base classes, which allows test designers
    #     to implement actual test methods in highly-parametrized base classes.)
    if isinstance(obj, type) and issubclass(obj, unittest.TestCase) and \
            self.getTestCaseNames(obj) and not isbase(obj, module):
        loaded_suite = self.loadTestsFromTestCase(obj)
        # ignore empty suites
        if loaded_suite.countTestCases():
            tests.append(loaded_suite)

where:

def isbase(cls, module):
    '''Returns True if cls is base class to any classes in module, else False.'''
    for name in dir(module):
        obj = getattr(module, name)
        if obj is not cls and isinstance(obj, type) and issubclass(obj, cls):
            return True
    return False

The parametrization I spoke of above is implemented by having each child class define it's fixture details (the parameters) and pass them to the base class TestCase ctor so that all of its common impl methods (the "fixturey" ones setUp*/tearDown*/cleanup* and the test methods themselves) have all the info that defines the now very specific fixture that that child TestCase class is to operate on.

For me, this was a temporary solution for quickly implementing some parametrized fixtures in unittest, since I plan to move my team's tests to pytest asap.