Creating a Celery worker using node.js

Question

Using node-celery, we can enable node to push Celery jobs to the task queue. How can we allow node to be a Celery worker and consume the queue?

Answer 1

Here is an approach from the Celery doc, by exposing REST api:

http://docs.celeryproject.org/en/latest/faq.html#is-celery-multilingual

Also, there’s another way to be language-independent, and that’s to use REST tasks, instead of your tasks being functions, they’re URLs. With this information you can even create simple web servers that enable preloading of code. Simply expose an endpoint that performs an operation, and create a task that just performs an HTTP request to that endpoint.

some example: http://ask.github.io/celery/cookbook/remote-tasks.html

Answer 2

For Celery if the end point is amqp. Checkout Celery.js Github any node process started as amqp consumer would work fine. For every other self.conf.backend_type types you can have varied consumer. Following example is merely for amqp.

One such example. The message below may be the Celery task object.

var amqp = require('amqp');
var connection = amqp.createConnection({ host: "localhost", port: 5672 });
connection.on('ready', function () {
  connection.queue("my_celery_queue", function(queue){
    queue.bind('#'); 
    queue.subscribe(function (message) {
      //eat your Celery work here
    })
  })
})