How to keep Laravel Queue system running on server

Question

I recently setup a Laravel Queue system. The basics are a cronjob calls a command which adds jobs to a queue and calls a second command which sends an email.

The sy

Answer 1

You should use linux supervisor

Installation is simple and on Ubuntu I can install it with following command:

apt-get install supervisor

Supervisor configuration files are located in /etc/supervisor/conf.d directory.

[program:email-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/laravel-example/artisan queue:work redis --queue=emailqueue --sleep=3 --tries=3
autostart=true
autorestart=true
user=forge
numprocs=2
redirect_stderr=true
stdout_logfile=/var/www/laravel-example//storage/logs/supervisord.log

For each process you should create a new process configuration file. With this configuration, listener will retry each job 3 times. Also Supervisor will restart listener if it fails or if system restarts.

Answer 2

For those who are already running NodeJS on their production environments. I use PM2 to manage app processes.

# install
npm install -g pm2

# in project dir with your CI or dev setup tool 
# --name gives task a name so that you can later manage it
# -- delimits arguments that get passed to the script
pm2 start artisan --interpreter php --name queue-worker -- queue:work --daemon

I use Vagrant in development and setup NodeJS and this process using only inline vagrant scripts.

When you use PM2 in development you can use one of the many watchers to manage the restart. Simply run pm2 restart queue-worker when you pick up a change. In production I don't recommend this approach, rather opt for a build tool that can follow this process.

# 1. stop pm task to ensure that no unexpected behaviour occurs during build
pm2 stop queue-worker
# 2. do your build tasks
...
# 3. restart queue so that it loads the new code
pm2 restart queue-worker

Answer 3

The best way is PM2 (Advanced, production process manager for Node.js) that you can monit your queues and see their's logs.

with command below in your project directory, run queue worker :

pm2 start artisan --name laravel-worker --interpreter php -- queue:work --daemon

Answer 4

Running

nohup php artisan queue:work --daemon &

Will prevent the command exiting when you log out.

The trailing ampersand (&) causes process start in the background, so you can continue to use the shell and do not have to wait until the script is finished.

See nohup

nohup - run a command immune to hangups, with output to a non-tty

This will output information to a file entitled nohup.out in the directory where you run the command. If you have no interest in the output you can redirect stdout and stderr to /dev/null, or similarly you could output it into your normal laravel log. For example

nohup php artisan queue:work --daemon > /dev/null 2>&1 &

nohup php artisan queue:work --daemon > app/storage/logs/laravel.log &

But you should also use something like Supervisord to ensure that the service remains running and is restarted after crashes/failures.

Answer 5

Since this was a Laravel-specific question, I thought I would suggest a Lravel-specific answer. Since you are already using cronjobs on this server, I would recommend that you set up the shell command as a recurring cronjob to always verify that the worker is running. You could either set up the shell command to run natively through cron on your server, or you could use the Laravel console kernel to manage the command and add logic, such as checking whether you already have a worker running and, if not, go ahead and start it back up.

Depending on how often you need to run your command, you could do this as infrequently as once a week, or even once a minute. This would give you the ability to make sure that your workers are continuously running, without having to add any overhead to your server, such as Supervisor. Giving permissions to a 3rd party package like supervisor is ok if you trust it, but if you can avoid needing to rely on it, you may want to consider this approach instead.

An example of using this to do what you want would be to have a cronjob that runs each hour. It would execute the following in sequential order from within a custom Laravel console command:

\Artisan::call('queue:restart');

\Artisan::call('queue:work --daemon');

Note that this applies for older versions of Laravel (up to 5.3) but I haven't tested on newer versions.

Answer 6

For CentOS7

yum install supervisor

Then create a file in /etc/supervisord.d/filename.ini With content

[program:laravel-worker]
command=/usr/bin/php /home/appuser/public_html/artisan queue:listen
process_name=%(program_name)s_%(process_num)02d
numprocs=5
priority=999
autostart=true
autorestart=true
startsecs=1
startretries=3
user=appuser
redirect_stderr=true
stdout_logfile=/path/logpath/artisan.log

Then start the supervisord service using

systemctl restart supervisord

Enable supervisord service to run on boot using

systemctl enable supervisord

Check if the service is running using

ps aux | grep artisan

You should see the process running if it was set up properly. Similar to the output below.

[root@server ~]# ps aux | grep artisan
appuser 17444  0.1  0.8 378656 31068 ?        S    12:43   0:05 /usr/bin/php /home/appuser/public_html/artisan queue:listen