Cloud Endpoints - Retrieving a single entity from datastore (by a property other than the helper methods provided by EndpointsModel)

Question

This question completely follows on from a related question I had asked (and was answered) here: Error when trying to retrieve a single entity

As I understand, to retrieve a single entity from the datastore using a property other than helper methods already provided (e.g. 'id') requires turning a simple data property into an EndpointsAliasProperty? If yes, how would I go about doing that? Or is it that we can only use 'id' (helper methods provided by EndpointsModel) and we cannot use any of the properties that we define (in this case 'title')?

Answer 1

The distinction between the custom EndpointsAliasPropertys and one of the data properties you defined is how they are used. They are all used to create a protorpc message, and that message is then converted into an EndpointsModel with your custom data in it. THIS is where the magic happens.

Breaking it down into steps:

1. You specify your data

from google.appengine.ext import ndb
from endpoints_proto_datastore.ndb import EndpointsModel

class MyModel(EndpointsModel):
    my_attr = ndb.StringProperty()

2. You pick your fields for your method

class MyApi(...):

    @MyModel.method(request_fields=('id', 'my_attr'), ...)
    def my_method(self, my_model_entity):
        ...

3. A protorpc message class is defined from your fields

>>> request_message_class = MyModel.ProtoModel(fields=('id', 'my_attr'))
>>> request_message_class
<class '.MyModelProto_id_my_attr'>
>>> for field in request_message_class.all_fields():
...   print field.name, ':', field.variant
...
id : INT64
my_attr : STRING

This happens every time a request is handled by a method decorated with @MyModel.method.

4. A request comes in your application and a message is created

Using the protorpc message class, a message instance is parsed from the JSON which gets passed along to your Endpoints SPI (which is created by endpoints.api_server).

When the request comes in to your protorpc.remote.Service it is decoded:

>>> from protorpc import remote
>>> protocols = remote.Protocols.get_default()
>>> json_protocol = protocols.lookup_by_content_type('application/json')
>>> request_message = json_protocol.decode_message(
...      request_message_class,
...      '{"id": 123, "my_attr": "some-string"}'
... )
>>> request_message
<MyModelProto_id_my_attr
 id: 123
 my_attr: u'some-string'>

5. The protorpc message is cast into a datastore model

entity = MyModel.FromMessage(request_message)

THIS is the step you really care about. The FromMessage class method (also provided as part of EndpointsModel) loops through all the fields

for field in sorted(request_message_class.all_fields(),
                    key=lambda field: field.number):

and for each field with a value set, turns the value into something to be added to the entity and separates based on whether the property is an EndpointsAliasProperty or not:

    if isinstance(value_property, EndpointsAliasProperty):
        alias_args.append((local_name, to_add))
    else:
        entity_kwargs[local_name] = to_add

After completing this loop, we have an ordered list alias_args of all key, value pairs and a dictionary entity_kwargs of the data attributes parsed from the message.

Using these, first a simple entity is created

entity = MyModel(**entity_kwargs)

and then each of the alias property values are set in order:

for name, value in alias_args:
    setattr(entity, name, value)

---

The extended behavior happens in setattr(entity, name, value). Since EndpointsAliasProperty is a subclass of property, it is a descriptor and it has a setter which can perform some custom behavior beyond simply setting a value.

For example, the id property is defined with:

@EndpointsAliasProperty(setter=IdSet, property_type=messages.IntegerField)
def id(self):

and the setter performs operations beyond simply setting data:

def IdSet(self, value):
    self.UpdateFromKey(ndb.Key(self.__class__, value))

This particular method attempts to retrieve the entity stored in the datastore using the id and patch in any values from the datastore that were not included in the entity parsed from the request.

---

If you wanted to do this for a field like my_attr, you would need to construct a custom query which could retrieve the item with that unique my_attr value (or fail if not exactly one such entity exists).

This is problematic and you'd be better off using a unique field like the key or ID used to store the entity in the datastore.

The keys with ancestors sample gives a great example of creating your own custom properties.

If you REALLY insist on using my_attr to retrieve an entity, you could do so using a different property name (since my_attr is already used for the data property) such as fromMyAttr:

class MyModel(EndpointsModel):
    def MyAttrSet(self, value):
        ...
    @EndpointsAliasProperty(setter=MyAttrSet)
    def fromMyAttr(self):
        ...

Here, the MyAttrSet instance method would form the query:

    def MyAttrSet(self, value):
        query = MyModel.query(MyModel.my_attr == value)
        results = query.fetch(2)

reject results that aren't unique for my_attr:

        if len(results) == 0:
            raise endpoints.NotFoundException('Not found.')
        if len(results) == 2:
            raise endpoints.BadRequestException('Colliding results.')

and copy over the values for the already stored entity if we do find a unique one:

        matching_entity = results[0]
        self._CopyFromEntity(matching_entity)
        self._from_datastore = True