When asyncio task gets stored after creation, exceptions from task get muted

Question

I was using asyncio for a project, and encountered this strange behavior.

import asyncio

def schedule_something():
    global f
    tsk = asyncio.async(do_s         


        

Answer 1

Thanks, @dano. Here's a drop-in replacement for asyncio.create_task that does this automatically -

def create_task(coro):
    task = asyncio.create_task(coro)
    return TaskWrapper(task)


class TaskWrapper:
    def __init__(self, task):
        self.task = task
        task.add_done_callback(self.on_task_done)

    def __getattr__(self, name):
        return getattr(self.task, name)

    def __await__(self):
        self.task.remove_done_callback(self.on_task_done)
        return self.task.__await__()

    def on_task_done(self, fut: asyncio.Future):
        if fut.cancelled() or not fut.done():
            return
        fut.result()

    def __str__(self):
        return f"TaskWrapper<task={self.task}>"

---

Updated version of the given example -

async def do_something():
    raise Exception()


async def schedule_something():
    global f
    tsk = create_task(do_something())
    f = tsk  # If this line is commented out, exceptions can be heard.


asyncio.run(schedule_something())

$ python test.py
Exception in callback TaskWrapper.on_task_done(<Task finishe...n=Exception()>)
handle: <Handle TaskWrapper.on_task_done(<Task finishe...n=Exception()>)>
Traceback (most recent call last):
  File "/Users/dev/.pyenv/versions/3.8.1/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/dev/Projects/dara/server/bot/async_util.py", line 21, in on_task_done
    fut.result()
  File "/Users/dev/Projects/dara/server/test.py", line 7, in do_something
    raise Exception()
Exception

Answer 2

This is because the exception only gets raised if the Task is destroyed without ever having its result retrieved. When you assigned the Task to a global variable, it will always have an active reference, and therefore never be destroyed. There's a docstring in asyncio/futures.py that goes into detail on this:

class _TracebackLogger:
    """Helper to log a traceback upon destruction if not cleared.

    This solves a nasty problem with Futures and Tasks that have an
    exception set: if nobody asks for the exception, the exception is
    never logged.  This violates the Zen of Python: 'Errors should
    never pass silently.  Unless explicitly silenced.'

    However, we don't want to log the exception as soon as
    set_exception() is called: if the calling code is written
    properly, it will get the exception and handle it properly.  But
    we *do* want to log it if result() or exception() was never called
    -- otherwise developers waste a lot of time wondering why their
    buggy code fails silently.

    An earlier attempt added a __del__() method to the Future class
    itself, but this backfired because the presence of __del__()
    prevents garbage collection from breaking cycles.  A way out of
    this catch-22 is to avoid having a __del__() method on the Future
    class itself, but instead to have a reference to a helper object
    with a __del__() method that logs the traceback, where we ensure
    that the helper object doesn't participate in cycles, and only the
    Future has a reference to it.

    The helper object is added when set_exception() is called.  When
    the Future is collected, and the helper is present, the helper
    object is also collected, and its __del__() method will log the
    traceback.  When the Future's result() or exception() method is
    called (and a helper object is present), it removes the the helper
    object, after calling its clear() method to prevent it from
    logging.

If you want to see/handle the exception, just use add_done_callback to handle the result of the task, and do whatever is necessary when you get an exception:

import asyncio

def handle_result(fut):
    if fut.exception():
        fut.result()  # This will raise the exception.

def schedule_something():
    global f
    tsk = asyncio.async(do_something())
    tsk.add_done_callback(handle_result)
    f = tsk

@asyncio.coroutine
def do_something():
    raise Exception()

loop = asyncio.get_event_loop()
loop.call_soon(schedule_something)
loop.run_forever()
loop.close()