Python unittest counting the number of tests

Question

It's my first time playing around with Python's unittest for an assignment in school. I basically have a Circle Object, where I am using pyunit to make sure the data is stored properly.

I noticed that Python only counts the number of methods as test cases as opposed to the number of assert statements.

For instance I want to test that methods are working correctly, Python only counts the following as 2 tests, despite having 4 assert statements. It really caught me off guard, as with Java's JUnit it will count the number of assert statements instead.

def test_xcrd(self): 
    self.assertTrue(self.point.xcrd() == 1) 
    self.assertFalse(self.point.xcrd() == 5)

def test_ycrd(self): 
    self.assertTrue(self.point.ycrd() == 2) 
    self.assertFalse(self.point.ycrd() == 10)

What's the "norm" in python? Should there only be one assert statement per method?

Answer 1

Python's unittest package allows you to structure your unit tests in separate methods like you are noticing. This is useful in cases where you want to tests things which are very closely related and don't require separate unit tests.

unittest tests start by subclassing unittest.Test, and then adding methods to this. So, you can add several layers separation between different unittests which are more less related.

An example from the Python Docs demonstrates what is considered to be best practice for Python unit tests:

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

There are a number of things you can observe here:

1. The three methods of TestStringMethods are the separate unittests.
2. test_isupper and test_split both contain two asserts, since they are very closely related. Adding separate tests for the two asserts present in test_isupper would be adding lots of bloat to the code, and it could lead to very weird problems.

For example, if str.isupper() would break in a weird way, the single unittest covering this single function would break. However, if the two tests for "FOO" and "Foo" were separate, one test might pass, and the other fail. So, testing the functionality of a single function is better kept in a single unittest with several asserts.

The same applies to the test_split method; checking that str.split() works and checking that it raises a TypeError are closely related, and should therefore best be kept close together in code as well.

So, to come back to your question: There can (and sometimes should) be more than one assert per method, since it leads to simpler and clearer code, and less confusion. To quote the "Zen of Python" (found by running import this in a python shell): "Simple is better than complex". So, keep your unittests simple and structured by grouping similar asserts in a single method.

Answer 2

The answer to your question '''What's the "norm" in python? Should there only be one assert statement per method?''' is "No". Some people might say 'yes' but CPython core developers, including me, routinely use multiple asserts in test methods. Take a look at the test_xyz files in Lib/test (if your installation includes that directory).

It is true that one method should test one unit or even one behavior of one unit.