In Google App Engine, how to I reduce memory consumption as I write a file out to the blobstore rather than exceed the soft memory limit?

Question

I\'m using the blobstore to backup and recovery entities in csv format. The process is working well for all of my smaller models. However, once I start to work on models with mo

Answer 1

It can possibly be a Time Exceed error, due to the limitation of the request to 30 secs. In my implementation in order to bypass it instead of having a webapp handler for the operation I am firing an event in the default queue. The cool thing about the queue is that it takes one line of code to invoke it, it has a 10 min time limit and if a task fails it retries before the time limit. I am not really sure if it will solve your problem but it worths giving a try.

from google.appengine.api import taskqueue
...
taskqueue.add("the url that invokes your method")

you can find more info about the queues here.

Or consider using a backend for serious computations and file operations.