Parameterizing a test using CppUnit

Question

My organization is using CppUnit and I am trying to run the same test using different parameters. Running a loop inside the test is not a good option as any failure will abort the test. I have looked at TestDecorator and TestCaller but neither seems to really fit. Code samples would be helpful.

Answer 1

It does not appear possible in CppUnit to parameterize a test case directly (see here and here). However, you do have a few options:

Use a RepeatedTest

You may be able to make some clever use of the built-in RepeatedTest decorator. This allows a test case to be run multiple times (though without parameterization).

I'll admit to never having used this myself, but perhaps you could have the RepeatedTest drive some gatekeeper function, which would (using a class static variable, perhaps?) pick a different input with every run. It would in turn call the true function you'd like to test with that value as input.

Use a TestCase subclass

One person on CppUnit's SourceForge page claims to have written a subclass of TestCase that will run a particular test an arbitrary number of times, although in a slightly different manner than the RepeatedTest class offers. Sadly, the poster simply described the motivation for creating the class, but did not provide the source code. There was, however, an offer to contact the individual for more details.

Use a simple helper function

The most straight-forward (but least automated) way to do this is to create a helper function that takes the parameter you'd like to pass on to your "real" function, and then have lots of individual test cases. Each test case would call your helper function with a different value.

---

If you choose either of the first two options listed above, I'd be interested in hearing about your experience.

Answer 2

class members : public CppUnit::TestFixture
{
    int i;
    float f;
};

class some_values : public members
{
    void setUp()
    {
        // initialization here
    }
};

class different_values : public members
{
    void setUp()
    {
        // different initialization here
    }
};

tempalte<class F>
class my_test : public F
{
    CPPUNIT_TEST_SUITE(my_test<F>);
    CPPUNIT_TEST(foo);
    CPPUNIT_TEST_SUITE_END();

    foo() {}
};

CPPUNIT_TEST_SUITE_REGISTRATION(my_test<some_values>);
CPPUNIT_TEST_SUITE_REGISTRATION(my_test<different_values>);

I don't know if that's considered kosher as per CppUnit's "preferred way of doing things" but that's the approach I'm taking now.

Answer 3

Upon of the suggestion of Marcin i've implemented some macros aiding to define parameterized CppUnit tests.

With this solution you just need to replace the old macros CPPUNIT_TEST_SUITE and CPPUNIT_TEST_SUITE_END within the class's header file:

CPPUNIT_PARAMETERIZED_TEST_SUITE(<TestSuiteClass>, <ParameterType>);

/*
 * put plain old tests here.
 */

CPPUNIT_PARAMETERIZED_TEST_SUITE_END();

In the implementation file you need to replace the old CPPUNIT_TEST_SUITE_REGISTRATION macro with:

CPPUNIT_PARAMETERIZED_TEST_SUITE_REGISTRATION ( <TestSuiteClass>, <ParameterType> )

These macros require you to implement the methods:

static std::vector parameters();
void testWithParameter(ParameterType& parameter);

• parameters(): Provides a vector with the parameters.
• testWithParameter(...): Is called for each parameter. This is where you implement your parameterized test.

A detailed explanation can be found here: http://brain-child.de/engineering/parameterizing-cppunit-tests

The german version can be found here: http://brain-child.de/engineering/parametrierbare-tests-cppunit

Answer 4

I'm not a C++ programmer but I can help with the unit-test concept:

Test-cases are meant to run isolated and with no dependency on external parameters. Additionally you should keep the number of test-cases down to the minimum which covers most of your code. There are cases, however (and I have already dealt with some), where some tests look the same, differing only by some minor parameters. The best bet then is to write a fixture which takes the parameter you're talking about, and then have one test-case for each of the parameters, calling the fixture with it. A generic example follows:

class MyTestCase

  # this is your fixture
  def check_special_condition(param)
    some
    complex
    tests
  end

  # these are your test-cases
  def test_1
    check_special_condition("value_1")
  end

  def test_2
    check_special_condition("value_2")
  end

end

Otherwise you're not writing true test-cases, because they're supposed to be reproducible without much knowledge from the one who is executing them. I imagine there are a handful of parameters which are all important as input to the tests. Then why not make each one explicit inside its own test-case? That's also the best way to document then, instead of writing a separate document to guide the programmer which will read the code years later.

Answer 5

This is a very old question, but I just needed to do something similar and came up with the following solution. I'm not 100% happy with it, but it seems to do the job quite well

1. Define a set of input parameters to a testing method. For example, let's say these are strings, so let's do:

   std::vector<std::string> testParameters = { "string1", "string2" };
   size_t testCounter = 0;
   

2. Implement a generic tester function, which with each invocation will take the next parameter from the test array, e.g.:

   void Test::genericTester()
   {
     const std::string &param = testParameters[testCounter++];
   
     // do something with param
   } 
   

3. In the test addTestToSuite() method declaration (hidden by the CPPUNIT macros) instead of (or next to) defining methods with the CPPUNIT_TEST macros, add code similar to this:

   CPPUNIT_TEST_SUITE(StatementTest);
   
   testCounter = 0;
   for (size_t i = 0; i < testParameters.size(); i++) {
     CPPUNIT_TEST_SUITE_ADD_TEST(
       ( new CPPUNIT_NS::TestCaller<TestFixtureType>(
                 // Here we use the parameter name as the unit test name.
                 // Of course, you can make test parameters more complex, 
                 // with test names as explicit fields for example.
                 context.getTestNameFor( testParamaters[i] ),
                 // Here we point to the generic tester function.
                 &TestFixtureType::genericTester,
                 context.makeFixture() ) ) );
   }
   
   CPPUNIT_TEST_SUITE_END();
   

This way we register genericTester() multiple times, one for each parameter, with a name specified. This seems to work for me quite well.

Hope this helps someone.

Answer 6

Based on consumerwhore answer, I ended up with a very nice approach where I can create multiple tests using a one-line registration macro with as many parameters I want.

Just define a parameter class:

class Param
{
public:
    Param( int param1, std::string param2 ) :
        m_param1( param1 ),
        m_param2( param2 )
    {
    }

    int m_param1;
    std::string m_param2;
};

Make your test fixture use it as "non-type template parameter" (I think that's how it's called):

template <Param& T>
class my_test : public CPPUNIT_NS::TestFixture
{
    CPPUNIT_TEST_SUITE(my_test<T>);
    CPPUNIT_TEST( doProcessingTest );
    CPPUNIT_TEST_SUITE_END();

    void doProcessingTest()
    {
        std::cout << "Testing with " << T.m_param1 << " and " << T.m_param2 << std::endl;
    };
};

Have a small macro creating a parameter and registering a new test fixture:

#define REGISTER_TEST_WITH_PARAMS( name, param1, param2 ) \
    Param name( param1, param2 ); \
    CPPUNIT_TEST_SUITE_REGISTRATION(my_test<name>);

Finally, add as many tests you want like that:

REGISTER_TEST_WITH_PARAMS( test1, 1, "foo" );
REGISTER_TEST_WITH_PARAMS( test2, 3, "bar" );

Executing this test will give you:

my_test<class Param test1>::doProcessingTestTesting with 1 and foo : OK
my_test<class Param test2>::doProcessingTestTesting with 3 and bar : OK
OK (2)
Test completed, after 0 second(s). Press enter to exit

Answer 7

The following class/helper macro pair works for my current use-cases. In your TestFixture subclass, just define a method which accepts one parameter and then add the test with PARAMETERISED_TEST(method_name, argument_type, argument_value).

#include <cppunit/extensions/HelperMacros.h>
#include <cppunit/ui/text/TestRunner.h>

template <class FixtureT, class ArgT>
class ParameterisedTest : public CppUnit::TestCase {
public:
  typedef void (FixtureT::*TestMethod)(ArgT);
  ParameterisedTest(std::string name, FixtureT* fix, TestMethod f, ArgT a) :
    CppUnit::TestCase(name), fixture(fix), func(f), arg(a) {
  }
  ParameterisedTest(const ParameterisedTest* other) = delete;
  ParameterisedTest& operator=(const ParameterisedTest& other) = delete;

  void runTest() {
    (fixture->*func)(arg);
  }
  void setUp() { 
    fixture->setUp(); 
  }
  void tearDown() { 
    fixture->tearDown(); 
  }
private:
  FixtureT* fixture;
  TestMethod func;
  ArgT arg;
};

#define PARAMETERISED_TEST(Method, ParamT, Param)           \
  CPPUNIT_TEST_SUITE_ADD_TEST((new ParameterisedTest<TestFixtureType, ParamT>(context.getTestNameFor(#Method #Param), \
                                          context.makeFixture(), \
                                          &TestFixtureType::Method, \
                                              Param)))

class FooTests : public CppUnit::TestFixture {
  CPPUNIT_TEST_SUITE(FooTests);
  PARAMETERISED_TEST(ParamTest, int, 0);
  PARAMETERISED_TEST(ParamTest, int, 1);
  PARAMETERISED_TEST(ParamTest, int, 2);
  CPPUNIT_TEST_SUITE_END();
public:
  void ParamTest(int i) {
    CPPUNIT_ASSERT(i > 0);
  }
};
CPPUNIT_TEST_SUITE_REGISTRATION(FooTests);

int main( int argc, char **argv)
{
  CppUnit::TextUi::TestRunner runner;
  CppUnit::TestFactoryRegistry &registry = CppUnit::TestFactoryRegistry::getRegistry();
  runner.addTest( registry.makeTest() );
  bool wasSuccessful = runner.run( "", false );
  return wasSuccessful;
}