Improve App Engine performance by reducing entity size

Question

The objective is to reduce the CPU cost and response time for a piece of code that runs very often and must db.get() several hundred keys each time.

Does this even work

Answer 1

To remove properties from an entity, you can change your Model to an Expando, and then use delattr. It's documented in the App Engine docs here:

http://code.google.com/intl/fr/appengine/articles/update_schema.html

Under the heading "Removing Deleted Properties from the Datastore"

Answer 2

if I want to reduce the size of my entities, is it necessary to migrate the old entities to ones with the new definition?

Yes. The GAE data store is just a big key-value store, that doesn't know anything about your model definitions. So the old values will be the old values until you put new values in!

Answer 3

To answer your questions in order:

• Yes, splitting up your model will reduce the fetch time, though probably not linearly. For a relatively small model like yours, the differences may not be huge. Large list properties are the leading cause of increased fetch time.
• Old properties will still be transferred when you fetch an entity after the change to the model, because the datastore has no knowledge of models.
• Also, however, deleted properties will still be stored even once you call .put(). Currently, there's two ways to eliminate the old properties: Replace all the existing entities with new ones, or use the lower-level api.datastore interface, which is dict-like and makes it easy to delete keys.