Using transactions on Django AppEngine

Question

The AppEngine documentation has a few examples of transactions, using the AppEngine native technique to make transacted queries on native objects.

Per the docs at http://www.allbuttonspressed.com/projects/django-nonrel [1], I'd like to use AppEngine transactions to query Django objects. Is this possible?

def txn():
    index = random.randint(0, NUM_SHARDS - 1)
    shard_name = "shard" + str(index)
    counter = SimpleCounterShard.objects.filter(name=shard_name)
    # Make counter if it doesn't exist 
    if not len(counter):
        counter = SimpleCounterShard(name=shard_name)
    counter.count += 1
    counter.put()
db.run_in_transaction(txn)

This currently fails with 'Only ancestor queries are allowed inside a transaction.' I understand this is asking me to do something involving an ancestor class, but I'm not sure what or why.

Any tips?

[1] "You can't use Django's transactions API. If your particular DB supports a special kind of transaction (e.g., run_in_transaction() on App Engine) you have to use the platform-specific functions."

Answer 1

As the error message suggests, only certain types of query are permissible inside a transaction on App Engine - specifically, ones that apply a .ancestor() filter. Queries such as the one you're attempting to execute can't be executed transactionally.

One option would be to execute the query outside the transaction, and pass the results in. It looks like you're trying to retrieve a specific shard of a sharded counter by name, though, and that ought to be possible without using a query at all, since it ought to be identified by its key name. Since I'm not familiar with Django's model API, though, I can't suggest how you'd do this in Django.

Answer 2

Nailer hit it on the head in his answer (sorry for the pun): DjangoAE does not support entity groups. However, there is unofficial support in this enterprising developer's branch of djangoappengine.

https://github.com/django-nonrel/djangoappengine/pull/10

The patch is not yet complete, but I plan on trying it out in the next few weeks and will update here.

Answer 3

A Django port of the AppEngine WebApp Sharded Counter example, including transactions on regular Django objects, lives in:

https://bitbucket.org/twanschik/sharded-counters-nonrel. Check out sharded_counters/models.py , which includes a single read/increment/write operation as discussed.

Specifically the @commit_locked decorator can be used to read/write/increment a Django model atomically.

Note however you are limited in the queries you can make inside a transaction: Django nonrel, as of Jan 2011, does not support entity groups, which is the cause of the error above.

Answer 4

A more elegant way is available:

from django.db.models import F
Accumulator.objects.filter(pk=1).update(counter=F('counter') + 5)

https://www.allbuttonspressed.com/blog/django/f-objects-and-queryset-update-support-in-djangoappengine

Another example is availabe here: https://www.allbuttonspressed.com/blog/django/2010/01/Sharding-with-Django-on-App-Engine#django-s-advantage

YouTubeVideo.objects.filter(pk=keyname).update(
    views_count=F('views_count')+1)