Asynchronous WSGI with Twisted

Question

I\'m building a web interface for a twisted application and would like to use WSGI rather than twisted.web directly (since the rest of the website is WSGI and I already have a s

Answer 1

In principle, WSGI is not intrinsically incompatible with asynchronous program design; in fact, PEP 333 goes to some considerable length to specify how servers, applications and middleware must behave to support that kind of thing.

At the heart of this is returning an iterator to the container. Every time an asynchronous wsgi app_iter is invoked, it would check on all of its pending asyncronous tasks (database connections, etcetera) and if any of them have data, the app_iter yields some data; otherwise it yields an empty string. To support this, a wsgi container would need to keep track of all of the in-flight requests, and iterate each of them in turn to get more data, in addition to servicing any other deferred work that it is responsible for.

In principle, very few wsgi apps or frameworks actually do this. almost invariably, wsgi frameworks block for all sorts of reasons; reading files from disk or loading data from a database for any reason at all (Most ORM's make this a tough problem to prevent.) Twisted's wsgi container operates under the assumption that since some wsgi apps block, that perhaps any wsgi app may block, and therefore always runs them in a thread.

There are two things you can do; either explore twisted's own web framework, which is fairly solid; or consider creating a wsgi wrapper for twisted outside of twisted's own container. Making sure the wsgi app is actually asyncronous is certainly a precondition of the latter, but wsgi itself is pretty simple, a thin wrapper over http, and so it should be easy enough.

Answer 2

Why do you want to use WSGI and do asynchronous things? The benefit of WSGI is that you can deploy your application on any WSGI container. If you start using Twisted APIs to do asynchronous things, then you can only deploy your application in Twisted's WSGI container.

You should probably just use Twisted Web without WSGI for your asynchronous code.