1. 技术文章
  2. Dynamically created method and decorator, got error 'functools.partial' object has no attribute '__module__'

Dynamically created method and decorator, got error 'functools.partial' object has no attribute '__module__'

由 六眼飞鱼酱① 提交于 2019-11-30 08:14:57

I also stumbled upon this, I was really surprised, for me the issue was that partial objects are missing certain attributes, specifically __module__ and __name__

Being that wraps by default uses functools.WRAPPER_ASSIGNMENTS to update attributes, which defaults to ('__module__', '__name__', '__doc__') in python 2.7.6 anyway, there are a couple ways of dealing with this ...

Update only present attributes ...

import functools
import itertools

def wraps_safely(obj, attr_names=functools.WRAPPER_ASSIGNMENTS):
    return wraps(obj, assigned=itertools.ifilter(functools.partial(hasattr, obj), attr_names))

>>> def foo():
...     """ Ubiquitous foo function ...."""
... 
>>> functools.wraps(partial(foo))(foo)()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/functools.py", line 33, in update_wrapper
setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'functools.partial' object has no attribute '__module__'
>>> wraps_safely(partial(foo))(foo)()
>>>

Here we simply filter out all those attribute which aren't present.

Another approach would be to strictly deal with partial objects only, you could fold wraps with singledispatch and create wrapped partial objects whose attributes would be taken from the deepest func attribute.

Something along the lines: 

import functools

def wraps_partial(wrapper, *args, **kwargs):
    """ Creates a callable object whose attributes will be set from the partials nested func attribute ..."""
    wrapper = wrapper.func
    while isinstance(wrapper, functools.partial):
        wrapper = wrapper.func
    return functools.wraps(wrapper, *args, **kwargs)

def foo():
    """ Foo function.
    :return: None """
    pass

>>> wraps_partial(partial(partial(foo)))(lambda : None).__doc__
' Foo Function, returns None '
>>> wraps_partial(partial(partial(foo)))(lambda : None).__name__
'foo'
>>> wraps_partial(partial(partial(foo)))(lambda : None)()
>>> pfoo = partial(partial(foo))
>>> @wraps_partial(pfoo)
... def not_foo():
...     """ Not Foo function ... """
... 
>>> not_foo.__doc__
' Foo Function, returns None '
>>> not_foo.__name__
'foo'
>>>

This is slightly better since now we can get the original functions docs which before defaulted to using the partial objects doc string.

This can be modified to only search if the current partial object doesn't already have the set attribute, which should be slightly faster when nesting many partial objects ...

UPDATE

It seems that python(CPython) 3 (at least 3.4.3) doesn't have this issue, since I don't know nor should I assume all versions of python 3 or other implementations such as Jython also share this issue here is another future ready approach 

from functools import wraps, partial, WRAPPER_ASSIGNMENTS

try:
    wraps(partial(wraps))(wraps)
except AttributeError:
    @wraps(wraps)
    def wraps(obj, attr_names=WRAPPER_ASSIGNMENTS, wraps=wraps):
        return wraps(obj, assigned=(name for name in attr_names if hasattr(obj, name)))

a couple things to note:

  • we define a new wraps function only if we fail to wrap a partial, in case future versions of python2 or other versions fix this issue.
  • we use the original wraps to copy the docs and other info
  • we don't use ifilter since it was removed in python3, I've timeit with and without ifilter but the results where inconclusive, at least in python (CPython) 2.7.6, the difference was marginal at best either way...

In Python 3.5 I have found that a reference to the original function is maintained in the partial. You can access it as .func:

from functools import partial

def a(b):
    print(b)


In[20]:  c=partial(a,5)

In[21]:  c.func.__module__
Out[21]: '__main__'

In[22]:  c.func.__name__
Out[22]: 'a'

If it is the case that it is caused by an issue with "wraps" in functools, there is nothing stopping you from writing your own partial that does not call wraps. According to the python documentation, this is a valid implementation of partial:

def partial(func, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = keywords.copy()
        newkeywords.update(fkeywords)
        return func(*(args + fargs), **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc

I stumbled upon this and thought would mention my workaround for this.

As rightly mentioned by @samy-vilar python3 doesn't have this issue. I have some code that uses functools.wrap and needs to run on python2 as well as python3.

For python2 we use functools32 which is backport of python3's functools for python2. wraps implementation of this package works perfect. Additionally it provides lru_cache which is available only in python3 functools.

import sys 

if sys.version[0] == '2':
   from functools32 import wraps
else:
   from functools import wraps

In our case I solved this by subclassing functools.partial:

class WrappablePartial(functools.partial):

    @property
    def __module__(self):
        return self.func.__module__

    @property
    def __name__(self):
        return "functools.partial({}, *{}, **{})".format(
            self.func.__name__,
            self.args,
            self.keywords
        )

    @property
    def __doc__(self):
        return self.func.__doc__

NB you could also make use of __getattr__ to redirect queries, but I figured that was actually less readable (and makes it more difficult to insert any useful meta-data as with __name__)

A pretty convenient solution for python 2.7 is described here: http://louistiao.me/posts/adding-name-and-doc-attributes-to-functoolspartial-objects/

Namely:

from functools import partial, update_wrapper

def wrapped_partial(func, *args, **kwargs):
    partial_func = partial(func, *args, **kwargs)
    update_wrapper(partial_func, func)

    return partial_func

This issue is fixed as of Python 2.7.11 (not sure which specific release it was fixed in). You can do functools.wraps on a functools.partial object in 2.7.11.

标签
python
google-app-engine
google-cloud-endpoints
functools
endpoints-proto-datastore
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!