iter() not working with datetime.now()

Question

A simple snippet in Python 3.6.1:

import datetime
j = iter(datetime.datetime.now, None)
next(j)

returns:

Traceback (most recent         


        

Answer 1

I assume you're using CPython and not another Python implementation. And I can reproduce the issue with CPython 3.6.1 (I don't have PyPy, Jython, IronPython, ... so I can't check these).

The offender in this case is the replacement of PyObject_Call with _PyObject_CallNoArg in the C equivalent of the callable_iterator.__next__ (your object is a callable_iterator) method.

The PyObject_Call does return a new datetime.datetime instance while _PyObject_CallNoArg returns NULL (which is roughly equivalent to an exception in Python).

Digging a bit through the CPython source code:

The _PyObject_CallNoArg is just a macro for _PyObject_FastCall which in turn is a macro for _PyObject_FastCallDict.

This _PyObject_FastCallDict function checks the type of the function (C-function or Python function or something else) and delegates to _PyCFunction_FastCallDict in this case because datetime.now is a C function.

Since datetime.datetime.now has the METH_FASTCALL flag it ends up in the fourth case but there _PyStack_UnpackDict returns NULL and the function is never even called.

I'll stop there and let the Python devs figure out what's wrong in there. @Martijn Pieters already filed a Bug report and they will fix it (I just hope they fix it soonish).

So it's a Bug they introduced in 3.6 and until it's fixed you need to make sure the method isn't a CFunction with the METH_FASTCALL flag. As workaround you can wrap it. Apart from the possibilities @Martijn Pieters mentioned there is also a simple:

def now():
    return datetime.datetime.now()

j = iter(now, None)
next(j)  # datetime.datetime(2017, 5, 31, 14, 23, 1, 95999)

Answer 2

This is definitely a bug introduced in Python 3.6.0b1. The iter() implementation recently switched to using _PyObject_FastCall() (an optimisation, see issue 27128), and it must be this call that is breaking this.

The same issue arrises with other C classmethod methods backed by Argument Clinic parsing:

>>> from asyncio import Task
>>> Task.all_tasks()
set()
>>> next(iter(Task.all_tasks, None))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

If you need a work-around, wrap the callable in a functools.partial() object:

from functools import partial

j = iter(partial(datetime.datetime.now), None)

I filed issue 30524 -- iter(classmethod, sentinel) broken for Argument Clinic class methods? with the Python project. The fix for this has landed and is part of 3.6.2rc1.