Twisted or Celery? Which is right for my application with lots of SOAP calls?

Question

I\'m writing a Python application that needs both concurrency and asynchronicity. I\'ve had a few recommendations each for Twisted and Celery, but I\'m having trouble determini

Answer 1

Is either Celery or Twisted a more generally appropriate framework here?

Depends on what you mean by "generally appropriate".

If they'll both solve the problem adequately, are there pros/cons to using one vs the other?

Not an exhaustive list.

Celery Pros:

• Ready-made distributed task queue, with rate-limiting, re-tries, remote workers
• Rapid development
• Comparatively shallow learning curve

Celery Cons:

• Heavyweight: multiple processes, external dependencies
• Have to run a message passing service
• Application "processes" will need to fit Celery's design

Twisted Pros:

• Lightweight: single process and not dependent on a message passing service
• Rapid development (for those familiar with it)
• Flexible
• Probably faster, no "internal" message passing required.

Twisted Cons:

• Steep learning curve
• Not necessarily as easy to add processing capacity later.

---

I'm familiar with both, and from what you've said, if it were me I'd pick Twisted.

I'd say you'll get it done quicker using Celery, but you'd learn more while doing it by using Twisted. If you have the time and inclination to follow the steep learning curve, I'd recommend you do this in Twisted.

Answer 2

Celery allows you to use asynchronous behavior of various async library like gevent and eventlet. So you can have best of both world.

Example using eventlet https://github.com/celery/celery/tree/master/examples/eventlet

Example using gevent https://github.com/celery/celery/tree/master/examples/gevent