google app engine ndb: put() and then query(), there is always one less item

Question

I wonder if anybody has encountered the strange problem on Google App Engine\'s NDB: after creating a new entity and saving it by put(); and then query()<

Answer 1

I solved this by making the appropriate query, creating the new model record, then appending it to my query before returning it. Modifying yours, it looks like:

class Item(ndb.Model):
    ...
    ...

items = Item.query().fetch()
length1 = len(items)
item = Item()
item.put()

appended_items = list()
for existing_item in items:
    appended_items.append(existing_item)

appended_items.append(item)
length2 = len(appendeditems)

In this case, appended_items has your query, plus the new element. The list generation is inefficient, but I'm a python/ndb noob and there's probably a way to get the Collection right out of the Query model, which would be much better.

Answer 2

The problem you are experiencing is that ndb provides "eventually consistant" data. With eventually consistent data, it typically takes ndb a few seconds to update the data so that you can use it later. This works for most applications, but if you need the data immediately after submitting a new entity, you will need "strongly consistent" data.

Google explains the difference in this great article: https://cloud.google.com/appengine/docs/python/datastore/structuring_for_strong_consistency

Answer 3

This is expected behaviour; your queries above are only eventually consistent. That is, you're not guaranteed to get the very latest results when querying.

You can get around this by using an ancestor query (see the link above). For your example, you'd need to give each of your items a parent entity and then use Item.query().ancestor(theParentEntity).fetch().

Answer 4

You could refer to their tutorial on nbd here.
They use a function to generate an ancestor key based on some Property of their ndb model. Depending on how you want your database, you could use a Property that is unique for several items, such as a User property, in a databse in which each user has several post let say. Or you could add a new dummy Property, say dummy = ndb.StringProperty() and for each item initialize that dummy with the same string, this way you'll get all entries that can later be filtered.

Answer 5

As @JesseRusak said, you need a Dummy ancestor to solve this little problem (I had the same problem that you recently).

But I didn't make a new DummyEntity, just only a DummyKey for a DummyAncestor. Try this for your problem:

class Item(ndb.Model): ... ... items = Item.query(ancestor=ndb.Key(Item, 'Items')).fetch() length1 = len(items) item = Item(parent=ndb.Key(Item, 'Items')) item.put() items = Item.query(ancestor=ndb.Key(Item, 'Items')).fetch() length2 = len(items)

At least in my case, the DummyAncestor worked.