Can celery celerybeat use a Database Scheduler without Django?

我们两清 提交于 2019-12-05 01:47:32

问题


I have a small infrastructure plan that does not include Django. But, because of my experience with Django, I really like Celery. All I really need is Redis + Celery to make my project. Instead of using the local filesystem, I'd like to keep everything in Redis. My current architecture uses Redis for everything until it is ready to dump the results to AWS S3. Admittedly I don't have a great reason for using Redis instead of the filesystem. I've just invested so much into architecting this with Docker and scalability in mind, it feels wrong not to.


回答1:


I was searching for a non-Django database scheduler too a while back, but it looked like there's nothing else. So I took the Django scheduler code and modified it to use SQLAlchemy. Should be even easier to make it use Redis instead.




回答2:


It turns out that you can!

First I created this little project from the tutorial on celeryproject.org.

That went great so I built a Dockerized demo as a proof of concept.

Things I learned from this project

  • Docker

    • using --link to create network connections between containers
    • running commands inside containers
  • Dockerfile

    • using FROM to build images iteratively
    • using official images
    • using CMD for images that "just work"
  • Celery

    • using Celery without Django
    • using Celerybeat without Django
    • using Redis as a queue broker
    • project layout
    • task naming requirements
  • Python

    • proper project layout for setuptools/setup.py
    • installation of project via pip
    • using entry_points to make console_scripts accessible
    • using setuid and setgid to de-escalate privileges for the celery deamon


来源:https://stackoverflow.com/questions/28760250/can-celery-celerybeat-use-a-database-scheduler-without-django

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!