unicode_literals and doctest in Python 2.7 AND Python 3.5

Question

Consider the following demo script:

# -*- coding: utf-8 -*-
from __future__ import division
from __future__ import unicode_literals

def myDivi():
    """
    This is a small demo that just returns the output of a divison.
    >>> myDivi()
    0.5
    """
    return 1/2

def myUnic():
    """
    This is a small demo that just returns a string.
    >>> myUnic()
    'abc'
    """
    return 'abc'

if __name__ == "__main__":
    import doctest
    extraglobs = {}
    doctest.testmod(extraglobs=extraglobs)

The doctest passes on Python 3.5, but fails on Python 2.7.9.
The strange thing is, the divison test works, but the unicode test fails.

I have seen various questions, including the following

• Multi version support for Python doctests
• Doctest not recognizing __future__.division
• Python: accept unicode strings as regular strings in doctests
• Python doctests and unicode
• Doctest fails due to unicode leading u

but they are all somewhat different (e.g. they are outdated (referring to Py 2.6 or Py 3.0), import statement is within the doctest instead of globally, use pytest instead of standard doctest, switch to different assert etc)
Still, I tried various alternatives based on these questions, including e.g.

if __name__ == "__main__":
    import doctest
    import __future__
    extraglobs = {'unicode_literals': __future__.unicode_literals}
    doctest.testmod(extraglobs=extraglobs)

or

def myUnic():
    """
    This is a small demo that just returns a string.
    >>> myUnic()
    u'abc' # doctest: +ALLOW_UNICODE
    """
    return 'abc'

but it still does not work, either on Python 2 or 3 or gives other errors.
Is there a way to make it pass on both 3.5+ AND 2.7.9+, without ugly hacks? I am also using these docstrings for generating documentation, so I would prefer to keep them more or less as they are.

Answer 1

In agreement with Martijn Pieters comments in Multi version support for Python doctests, I suggest to rely on testing using some real unit test framework.

You may still use the doctest strings, because they may be nice for documentation. Think for the future, and write them for Python 3. The same time, write unit tests for another unit-testing framework. Do not rely on doctest for Python 2 version of your application/module.

Answer 2

This does the job with pure doctest:

if __name__ == "__main__":
    import doctest, sys, logging, re
    from doctest import DocTestFinder, DocTestRunner
    # Support print in doctests.
    L_ = logging.getLogger(":")
    logging.basicConfig(level=logging.DEBUG)
    pr = print = lambda *xs: L_.debug(" ".join(repr(x) for x in xs))

    # Make doctest think u"" and "" is the same.
    class Py23DocChecker(doctest.OutputChecker, object):
        RE = re.compile(r"(\W|^)[uU]([rR]?[\'\"])", re.UNICODE)

        def remove_u(self, want, got):
            if sys.version_info[0] < 3:
                return (re.sub(self.RE, r'\1\2', want), re.sub(
                    self.RE, r'\1\2', got))
            else:
                return want, got

        def check_output(self, want, got, optionflags):
            want, got = self.remove_u(want, got)
            return super(Py23DocChecker, self).check_output(
                want, got, optionflags)

        def output_difference(self, example, got, optionflags):
            example.want, got = self.remove_u(example.want, got)
            return super(Py23DocChecker, self).output_difference(
                example, got, optionflags)

    finder = DocTestFinder()
    runner = DocTestRunner(checker=Py23DocChecker())
    for test in finder.find(sys.modules.get('__main__')):
        runner.run(test)
    runner.summarize()

• Treats u"foo" the same as "foo"
• Use print("foo") or pr("foo") to debug when running doctests. This only works if you are only using print for debugging purposes.

I've stolen most of it from places I don't remember. Thanks to these unsung heroes of the internet.