Python assert — improved introspection of failure?

Question

This is a rather useless assertion error; it does not tell the values of the expression involved (assume constants used are actually variable names):

$ python -c "assert 6-(3*2)"
[...]
AssertionError

Is there a better assert implementation in Python that is more fancy? It must not introduce additional overhead over execution (except when assert fails) .. and must turn off if -O flag is used.

Edit: I know about assert's second argument as a string. I don't want to write one .. as that is encoded in the expression that is being asserted. DRY (Don't Repeat Yourself).

Answer 1

Install your of function as sys.excepthook -- see the docs. Your function, if the second argument is AssertionError, can introspect to your heart's contents; in particular, through the third argument, the traceback, it can get the frame and exact spot in which the assert failed, getting the failing exception through the source or bytecode, the value of all relevant variables, etc. Module inspect helps.

Doing it in full generality is quite a piece of work, but depending on what constraints you're willing to accept in how you write your asserts it can be lightened substantially (e.g. restricting them to only local or global variables makes introspection easier than if nonlocal variables of a closure could be involved, and so forth).

Answer 2

You can attach a message to an assert:

assert 6-(3*2), "always fails"

The message can also be built dynamically:

assert x != 0, "x is not equal to zero (%d)" % x

See The assert statement in the Python documentation for more information.

Answer 3

As @Mark Rushakoff said nose can evaluate failed asserts. It works on the standard assert too.

# test_error_reporting.py
def test():
    a,b,c = 6, 2, 3
    assert a - b*c

nosetests' help:

$ nosetests --help|grep -B2 assert
  -d, --detailed-errors, --failure-detail
                        Add detail to error output by attempting to evaluate
                        failed asserts [NOSE_DETAILED_ERRORS]

Example:

$ nosetests -d
F
======================================================================
FAIL: test_error_reporting.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "..snip../site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "..snip../test_error_reporting.py", line 3, in test
    assert a - b*c
AssertionError:
    6,2,3 = 6, 2, 3
>>  assert 6 - 2*3


----------------------------------------------------------------------
Ran 1 test in 0.089s

FAILED (failures=1)

Answer 4

The nose testing suite applies introspection to asserts.

However, AFAICT, you have to call their asserts to get the introspection:

import nose
def test1():
    nose.tools.assert_equal(6, 5+2)

results in

C:\temp\py>C:\Python26\Scripts\nosetests.exe -d test.py
F
======================================================================
FAIL: test.test1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\nose-0.11.1-py2.6.egg\nose\case.py", line
183, in runTest
    self.test(*self.arg)
  File "C:\temp\py\test.py", line 3, in test1
    nose.tools.assert_equal(6, 5+2)
AssertionError: 6 != 7
>>  raise self.failureException, \
          (None or '%r != %r' % (6, 7))

Notice the AssertionError there. When my line was just assert 6 == 5+2, I would get:

C:\temp\py>C:\Python26\Scripts\nosetests.exe -d test.py
F
======================================================================
FAIL: test.test1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\nose-0.11.1-py2.6.egg\nose\case.py", line
183, in runTest
    self.test(*self.arg)
  File "C:\temp\py\test.py", line 2, in test1
    assert 6 == 5 + 2
AssertionError:
>>  assert 6 == 5 + 2

Also, I'm not sure offhand if their asserts are skipped with -O, but that would be a very quick check.

Answer 5

Add a message to your assertion, which will be displayed if the assertion fails:

$ python -c "assert 6-(3*2), '6-(3*2)'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AssertionError: 6-(3*2)

The only way I can think of to provide this automatically would be to contain the assertion in a procedure call, and then inspect the stack to get the source code for that line. The additional call would, unfortunately, introduce overhead into the test and would not be disabled with -O.

Answer 6

It sounds like what you really want to do is to set up a debugger breakpoint just before the assert and inspect from your favorite debugger as much as you like.

Answer 7

I coded a replacement for sys.excepthook (which is called for any unhandled exception) which is a bit more fancy than the standard one. It will analyze the line where the exception occured and print all variables which are referred to in this line (it does not print all local variables because that might be too much noise - also, maybe the important var is global or so).

I called it py_better_exchook (perfect name) and it's here.

Example file:

a = 6

def test():
    unrelated_var = 43
    b,c = 2, 3
    assert a - b*c

import better_exchook
better_exchook.install()

test()

Output:

$ python test_error_reporting.py 
EXCEPTION
Traceback (most recent call last):
  File "test_error_reporting.py", line 12, in <module>
    line: test()
    locals:
      test = <local> <function test at 0x7fd91b1a05f0>
  File "test_error_reporting.py", line 7, in test
    line: assert a - b*c
    locals:
      a = <global> 6
      b = <local> 2
      c = <local> 3
AssertionError

There are a few other alternatives:

• (Presented here) https://github.com/albertz/py_better_exchook/
• https://github.com/patrys/great-justice
• Nose does something similar for assertion failures, see here.
• IPython has something similar (this). Do this: from IPython.core import ultratb; sys.excepthook = ultratb.VerboseTB().
• Ka-Ping Yee's "cgitb.py", which is part of Python, see here, code here.