Running multiple Laravel queue workers using Supervisor

Question

I using Laravel queues using a database driver and supervisor to keep a queue worker running all the time:

[program:laravel_queue]
command=php artisan queue:         


        

Answer 1

In supervisor you specify the amount of processes with the parameter numprocs, so you can add a line to your script that says:

numprocs=5

Now, you can do some clever things, for example if only some of the processes that runs on the queues takes too long, you can create a different set of queues processes to work those and other set for the light processes. To achieve that you can create a supervisor configuration with one queue name like --queue=longprocess and another one with --queue=lightprocess and in your program you dispatch the job in the corresponding queue, that way, long processes won't delay short processes.

You can also specify queue priorities within one supervisor configuration file such as --queue=lightprocess,longprocess. That way, your worker(s) will first look for lightprocess before running longprocess.

To answer your second question, no, in terms of supervisor all processes are running, it doesn't knows if the queue is busy or idle, therefore it can't kill processes and create them based on their use, so no, you can't have a dynamic configuration of creating more processes only when the ones you have are busy.

A note, if you assign more than 1 numprocs, you must add the number of process to the name. According to supervisor configuration docs:

Supervisor will start as many instances of this program as named by numprocs. Note that if numprocs > 1, the process_name expression must include %(process_num)s (or any other valid Python string expression that includes process_num) within it.

Supervisor configuration docs: http://supervisord.org/configuration.html

Answer 2

From Laravel docs you can use numprocs=3 to spawn 3 processes.

And also specify a queue: command=php artisan queue:listen --queue=myqueue --tries=5

Answer 3

In supervisor config file. Add following code

process_name=%(program_name)s_%(process_num)02d
numprocs=8

This will create the 8 different processes of your program. You just need to change the value of numprocs=8 if you want.

You can check how many processes are running with sudo supervisorctl status