Is concurrency possible in tornado?

Question

I understand tornado is a single threaded and non-Blocking server, hence requests are handled sequentially (except when using event driven approach for IO operation).

<

Answer 1

If you are truly going to be dealing with multiple simultaneous requests that are compute-bound, and you want to do it in Python, then you need a multi-process server, not multi-threaded. CPython has Global Interpreter Lock (GIL) that prevents more than one thread from executing python bytecode at the same time.

Most web applications do very little computation, and instead are waiting for I/O, either from the database, or the disk, or from services on other servers. Be sure you need to handle compute-bound requests before discarding Tornado.

Answer 2

The answer to your question really depends on how long these compute-bound threads will be running for. If they're short running, and the rate of processing them at least matches the rate at which they arrive, then Tornado will be fine. It is truly single-threaded, but it does scale very well.

If your compute-bound requests are long running, then using a threaded server won't necessarily help because, as Ned Batchelder already pointed out, the GIL will be a bottleneck.

If you're able to relax the restriction of having the same memory space across all requests then you might consider running Tornado with PyZMQ, as it provides a way of running multiple Tornado back-ends, fronted by a single Tornado instance. This allows you to continue to use Tornado for the entire solution. See PyZMQ's web.zmqweb module for more information.