Override attribute access precedence having a data descriptor

问题

I have a bunch of instances of a MongoEngine model. And the profiler shows that a lot of time is spent in __get__ method of MongoEngine model fields:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.066    0.066   26.525   26.525 entity.py:60(list)
     2198    0.277    0.000   25.260    0.011 ***.py:96(***)
    45603    0.822    0.000   24.832    0.001 ***.py:105(***)
   285055    2.732    0.000   21.417    0.000 fields.py:189(__get__)
   444491    2.643    0.000   17.476    0.000 dereference.py:12(__call__)

As these model instances are read-only I want to make them to use simple Python data types. But I am not able to replace attributes:

> .../course_cache.py(339)_patch_me_model_instance()
    338         import ipdb; ipdb.set_trace()
--> 339         return obj
    340 

{1890: 0.6, 1891: 0.4, 1892: 0.6, 1893: 0.4, 1894: 0.2, 1895: 0.8}
ipdb> pinfo obj.tasks
Type:        BaseDict
String form: {1890: 0.6, 1891: 0.4, 1892: 0.6, 1893: 0.4, 1894: 0.2, 1895: 0.8}
Namespace:   Locals
Length:      6
File:        .../local/lib/python2.7/site-packages/mongoengine/base/datastructures.py
Docstring:
A special dict so we can watch any changes

ipdb> obj.__dict__['tasks'] = dict(obj.tasks)
ipdb> pinfo obj.tasks
Type:        BaseDict
String form: {1890: 0.6, 1891: 0.4, 1892: 0.6, 1893: 0.4, 1894: 0.2, 1895: 0.8}
Namespace:   Locals
Length:      6
File:        .../local/lib/python2.7/site-packages/mongoengine/base/datastructures.py
Docstring:
A special dict so we can watch any changes

This is described in the docs:

If an instance’s dictionary has an entry with the same name as a data descriptor, the data descriptor takes precedence.

But is there a way to override precedence for the attributes which are data descriptors without patching the model (removing the descriptor or adding __getattribute__)?

回答1:

Here is my solution which seems to work:

def _mock_me_instance(self, obj):
    """Patch a MongoEngine model instance to not use descriptors
    to access field values.
    """
    # copy class
    Model = type(obj.__class__.__name__, obj.__class__.__bases__,
                 dict(obj.__class__.__dict__))
    # add the original model to the base classes, so that isinstance() can work
    Model.__bases__ = (self.__class__,) + Model.__bases__
    # replace descriptor so that values from __dict__ can be seen
    for field_name in obj._fields:
        setattr(Model, field_name, None)

    obj.__dict__.update(obj.__dict__['_data'])
    obj.__class__ = Model  # replace the class
    return obj

How it works

I create a copy of the class whose instance the object is.
Put the original model in the base classes, so that isinstance and issubclass can work.
I copy the data into the object's __dict__. The values are ignored for now, because we have data descriptors with the same name which override access to the data.
I assign None to the descriptors in the class (MongoEngine model fields in my case). None is not a descriptor, so values from __dict__ of the object are seen now.
I replace object's class

The speed gain is ~600% (was 17 seconds, now it's 3).

来源：https://stackoverflow.com/questions/30342212/override-attribute-access-precedence-having-a-data-descriptor

标签

python

mongoengine