How to do OAuth-requiring operations in a GAE Task Queue?

Question

I have a simple Google App Engine app that includes a /update page which updates a YouTube playlist. It looks like this:

class UpdatePage(webapp2.RequestHandler):

    @decorator.oauth_required
    def get(self):
        update_result = self.update_playlist()
        ...

routes = [('/update', UpdatePage),
          (decorator.callback_path, decorator.callback_handler())]
app = webapp2.WSGIApplication(routes, debug=True)

It works as expected and the update_playlist() method does its job, but it turns out that under some circumstances it can run for a pretty long time, resulting in a DeadlineExceededError. So after reading about the available options, I figured the Task Queue API is the way to go (right?) and I'm trying to use it, following the Using Push Queues in Python guide.

→ In short, I split UpdatePage into UpdatePageHandler + UpdatePageWorker:

class UpdateHandlerPage(webapp2.RequestHandler):

    @decorator.oauth_required
    def get(self):
        taskqueue.add(url='/updateworker')

class UpdateWorkerPage(webapp2.RequestHandler):

    def post(self):
        update_result = self.update_playlist()
        ...

routes = [('/update', UpdateHandlerPage),
          ('/updateworker', UpdateWorkerPage),
          (decorator.callback_path, decorator.callback_handler())]
app = webapp2.WSGIApplication(routes, debug=True)

Unfortunately, after doing the split it seems my OAuth2 decorator no longer does its job:

INFO     2013-05-30 17:08:53,971 discovery.py:709] URL being requested: https://www.googleapis.com/youtube/v3/playlists?alt=json&part=snippet%2Cstatus
WARNING  2013-05-30 17:08:53,975 urlfetch_stub.py:480] Stripped prohibited headers from URLFetch request: ['content-length']
INFO     2013-05-30 17:08:54,351 client.py:493] Refreshing due to a 401
INFO     2013-05-30 17:08:54,361 appengine.py:276] make: Got type <class 'google.appengine.api.datastore_types.Blob'>
INFO     2013-05-30 17:08:54,363 appengine.py:289] validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
INFO     2013-05-30 17:08:54,364 client.py:680] Refreshing access_token
INFO     2013-05-30 17:08:54,746 client.py:699] Failed to retrieve access token: {
  "error" : "invalid_grant"
}
INFO     2013-05-30 17:08:54,757 appengine.py:276] make: Got type <class 'google.appengine.api.datastore_types.Blob'>
INFO     2013-05-30 17:08:54,759 appengine.py:289] validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
INFO     2013-05-30 17:08:54,761 appengine.py:289] validate: Got type <class 'oauth2client.client.OAuth2Credentials'>
INFO     2013-05-30 17:08:54,762 appengine.py:265] get: Got type <class 'oauth2client.appengine.CredentialsModel'>

And if instead of decorating UpdateHandlerPage:get I decorate UpdateWorkerPage:post, I get an infinite failure loop instead:

INFO     2013-05-30 17:24:31,307 discovery.py:190] URL being requested: https://www.googleapis.com/discovery/v1/apis/youtube/v3/rest?userIp=127.0.0.1
INFO     2013-05-30 17:24:34,960 dev_appserver.py:3105] "GET /update HTTP/1.1" 200 -
INFO     2013-05-30 17:24:35,060 dev_appserver.py:3105] "POST /updateworker HTTP/1.1" 302 -
WARNING  2013-05-30 17:24:35,065 taskqueue_stub.py:1980] Task task1 failed to execute. This task will retry in 0.100 seconds
INFO     2013-05-30 17:24:35,240 dev_appserver.py:3105] "POST /updateworker HTTP/1.1" 302 -
WARNING  2013-05-30 17:24:35,245 taskqueue_stub.py:1980] Task task1 failed to execute. This task will retry in 0.200 seconds
...

What can I do? Thanks for your help!

Answer 1

Since the Task Queue task will be spawned by your application, none of the headers from your original request will be sent through. In particular, the Cookies header identifying your user via the SACSID cookie for your application (provided by the App Engine Users API).

UPDATE: (This was added after the original post.) As a result of no cookies, the SACSID cookie identifying the user won't be there, hence the decorator.oauth_required designation will force a redirect (which is HTTP 302) EVERY time the cron job runs.

Instead of trying to get the current user from the decorator, you are better off passing along the App Engine User ID to your task. First get the current user (within your decorated method):

from google.appengine.api import users
# Guaranteed not to be None by the decorator
current_user = users.get_current_user()

and then pass along the App Engine User ID in the task

import urllib
query_string = urllib.urlencode({'user_id': current_user.user_id()})
taskqueue.add(url='/updateworker?' + query_string)

Then within your task, you can retrieve grab that user_id

# This is the 'user_id' you appended in the query string
user_id = self.request.get('user_id')

and use it to get that user's credentials as is done in the decorator:

from oauth2client.appengine import CredentialsModel
from oauth2client.appengine import StorageByKeyName
# This assumes you are using the defaults for OAuth2Decorator,
# which your above code is
credentials = StorageByKeyName(
    CredentialsModel, user_id, 'credentials').get()