Python doctest: skip a test conditionally

Question

I know how to skip a doctest using # doctest: +SKIP, but I can\'t figure out how to skip a test sometimes, based on a runtime condition. For example:

Answer 1

You can return a special value if you don't want the output to be tested. Let's call _skip this special value:

• if the appropriate flag is set and if the value you got is _skip, then the test is a success no matter what was expected
• otherwise (ie. no flag or a normal return value), perform the test a usual.

You need a custom OutputChecker:

_skip = object()
COND_SKIP = doctest.register_optionflag('COND_SKIP')

class CondSkipChecker(doctest.OutputChecker):
    def check_output(self, want, got, optionflags):
        if optionflags & COND_SKIP and got.strip() == str(_skip):
            return True
        else:
            return super(CondSkipChecker, self).check_output(want, got, optionflags)

Here's a proof of concept (the doctest API is a bit cumbersome: one woukld like to use testmod̀ with a checker argument):

"""
>>> 1 if True else _skip
2
>>> 1 if False else _skip
2
>>> 1 if True else _skip # doctest: +COND_SKIP
2
>>> 1 if False else _skip # doctest: +COND_SKIP
2
"""

import doctest, sys
_skip = object()
COND_SKIP = doctest.register_optionflag('COND_SKIP')

class CondSkipChecker(doctest.OutputChecker):
    def check_output(self, want, got, optionflags):
        if optionflags & COND_SKIP and got.strip() == str(_skip):
            return True
        else:
            return super(CondSkipChecker, self).check_output(want, got, optionflags)

finder = doctest.DocTestFinder()
runner = doctest.DocTestRunner(CondSkipChecker())
m = sys.modules.get('__main__')
for test in finder.find(m, m.__name__):
    runner.run(test)
print(runner.summarize())

Output:

**********************************************************************
File "temp.py", line 2, in __main__
Failed example:
    1 if True else _skip
Expected:
    2
Got:
    1
**********************************************************************
File "temp.py", line 4, in __main__
Failed example:
    1 if False else _skip
Expected:
    2
Got:
    <object object at 0x0033B8A8>
**********************************************************************
File "temp.py", line 6, in __main__
Failed example:
    1 if True else _skip # doctest: +COND_SKIP
Expected:
    2
Got:
    1
**********************************************************************
1 items had failures:
   3 of   4 in __main__
***Test Failed*** 3 failures.
TestResults(failed=3, attempted=4)

The two tests without doctest annotation fail as expected. Note: you can easily add a warning if _skip is used without the COND_SKIP flag.

The third test fails (got 1 vs expected 2), but the fourth is a success (got ̀_skip + COND_SKIP vs anything is okay).