Delayed job exclude queue

问题

I have a delayed job queue which contains particularly slow running tasks, which I want to be crunched by its own set of dedicated workers, so there is less risk it'll bottleneck the rest of the worker pipeline.

RAILS_ENV=production script/delayed_job --queue=super_slow_stuff start

However I then also want a general worker pool for all other queues, hopefully without having to specify them seperately (as their names etc are often changed/added too). Something akin to:

RAILS_ENV=production script/delayed_job --except-queue=super_slow_stuff start

I could use the wildcard * charecter but I imagine this would cause the second worker to pickup the super slow jobs too?

Any suggestions on this?

回答1:

you can define a global constant for your app with all queues.

QUEUES={
  mailers: 'mailers',
  etc..
}

then use this constant in yours delay method calls

object.delay(queue: QUEUES[:mailers]).do_something

and try to build delayed_job_args dinamically

system("RAILS_ENV=production script/delayed_job --pool=super_slow_stuff --pool:#{(QUEUES.values-[super_slow_stuff]).join(',')}:number_of_workers start")

回答2:

Unfortunately this functionality not realized in delayed jobs.
See:
https://github.com/collectiveidea/delayed_job/pull/466
https://github.com/collectiveidea/delayed_job/pull/901

You may fork delayed jobs repository and apply the simple patches from https://github.com/collectiveidea/delayed_job/pull/466.
Then use your GitHub repo, but please vote into https://github.com/collectiveidea/delayed_job/pull/466 to make it merged finally into upstream.

Update: I wrote option to exclude queues for myself. It is in (exclude_queues) branch: https://github.com/one-more-alex/delayed_job/tree/exclude_queues
https://github.com/one-more-alex/delayed_job_active_record/tree/exclude_queues

Options description included in Readme.md

Parts about exclusion.

# Option --exclude-specified-queues will do inverse of queues processing by skipping onces from --queue, --queues.
# If both --pool=* --exclude-specified-queues given, no exclusions will by applied on "*".

If EXCLUDE_SPECIFIED_QUEUES set to YES, then queues defined by QUEUE, QUEUES will be skipped instead. See opton --exclude-specified-queues description for specal case of queue "*"

If answer strictly on question, the calling of general worker will be like:

RAILS_ENV=production script/delayed_job --queue=super_slow_stuff --exclude-specified-queues start

Warning
Please not, that am not going to support DelayedJobs and code placed "as is" in hope it will be useful. Corresponding pull request was made by me https://github.com/collectiveidea/delayed_job/pull/1019

Also for Active Record backend: https://github.com/collectiveidea/delayed_job_active_record/pull/151

Only ActiveRecord backend supported.

来源：https://stackoverflow.com/questions/44808629/delayed-job-exclude-queue