Rails + Puma + Nginx Every couple of days Bad Gateway 502

Question

I have a rails app running on Nginx with Puma and like clockwork, every couple of days the app goes down with a 502 Bad Gateway error.

My nginx log contains lots of errors like this:

2015/07/23 14:43:49 [error] 14044#0: *7036 connect() to unix:///var/www/myapp/myapp_app.sock failed (111: Connection refused) while connecting to upstream, client: 12.123.12.12, server: myapp.com, request: "GET /arrangements HTTP/1.1", upstream: "http://unix:///var/www/myapp/myapp_app.sock:/arrangements", host: "myapp.com", referrer: "http://myapp.com/arrangements"

I have to restart Puma and everything works again...for a couple days.

Any ideas how I can troubleshoot this? I'm newer to nginx and puma.

/etc/nginx/sites-enabled/myapp.com

upstream myapp {
                server unix:///var/www/myapp/myapp_app.sock;
        }
        server {
                listen 80;
                server_name myapp.com;
                root /var/www/myapp/current/public;
                client_max_body_size 20M;

                location ~ \.php$ {
                        try_files $uri =404;
                        fastcgi_pass unix:/var/run/php5-fpm.sock;
                        fastcgi_index index.php;
                        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                        include fastcgi_params;
                        allow all;
                        satisfy any;
                }

                location / {
                        proxy_pass http://myapp; # match the name of upstream directive which is defined above
                        proxy_set_header Host $host;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                }
                location ~* ^/assets/ {
                        # Per RFC2616 - 1 year maximum expiry
                        expires 1y;
                        add_header Cache-Control public;

                        # Some browsers still send conditional-GET requests if there's a
                        # Last-Modified header or an ETag header even if they haven't
                        # reached the expiry date sent in the Expires header.
                        add_header Last-Modified "";
                        add_header ETag "";
                        break;
                }
        }

Answer 1

The DigitalOcean network team has identified an issue with firmware running on a number of network switches within NYC3. This issue is causing intermittent loss of connectivity to customer droplets.

While the issue has been confirmed only in a subset of racks, we will be upgrading all switches running the affected firmware in NYC3. This maintenance will result in approximately ten minutes of downtime per rack at some point within the maintenance window as individual switches are upgraded.

Maintenance window: 2015-08-27 22:00 EDT - 2015-08-28 02:00 EDT 2015-08-28 02:00 UTC - 2015-08-28 06:00 UTC

We apologize for the inconvenience and appreciate your patience as we work to improve the reliability of our network.

---

I would give it a day or two and see if the problem you're having recurs, or simply disappears on its own.

Added/Edited

P.S. I just noticed a detail on the email,

Affected Droplets: railsbox00

if you're getting the e-mails, then your droplet is affected by the firmware problem. Check your emails and see if they list your VPS; it's at the bottom of the email.

Answer 2

I don't know if this question is still relevant, but what helped me greatly with this exact problem was to move the actual location of the puma.sock file to another directory. I picked the /tmp directory.

The socket used to be on a drive that was NFS mounted to another server, and I believe that that was the problem - some hiccups in the network here and there. I'm not sure what it was exactly but since I moved the puma.sock to /tmp all problems disappeared. For me.