Nginx uwsgi (104: Connection reset by peer) while reading response header from upstream

Question

Environment is Nginx + uwsgi.

Getting a 502 bad gateway error from Nginx on certain GET requests. Seems to be related to the length of the URL. In our particular cas

Answer 1

--post-buffering 32768 worked for me as suggested (and discouraged) here NGINX + uWSGI Connection Reset by Peer

I don't have time to investigate it further at the moment (quick prototyping mode :), but since it took me a lot of time to find this hack, it might be worth posting here.

Answer 2

This can happen if your request/response headers are quite large.

To fix it, in /etc/uwsgi/apps-available/your-app.ini add buffer-size=65535

Answer 3

It doesn't come up occasionally.

I guess the most possible reason of that is the size of your php-fpm.log is oversize. Try to change your log_level to upper level in php-fpm.conf and clear the logs.

Anyway, it works for me.

Answer 4

We just need to increase the attribute "output_buffering" value, in php.ini, to a greater value like 65535 or another appropriate value.

Answer 5

When we receive a message like (104: Connection reset by peer) while reading response header from upstream, most often, we could blame the upstream side of this kind of error.

As described, the connection was reset by the upstream peer, not by nginx itself. Nginx as a client can barely do anything to make it right.

I'm suspecting if modifying buffer-size will do the magic. Basically the command changes the buffer size where response headers are cached. This would take effect when the response header is too big, of which case we receive a message saying upstream sent too big header while reading response header from upstream, and that is totally different thing from connection reset by peer.

Since this kind of error is trigger randomly, I would suggest you check whether nginx uses keepalive when talking to upstreams. If this was the case, the connection might be reset by upstream server when the idle timed out whereas nginx had no idea that the connection had been dropped, hence forwarding the request using the same connection.

There's no elegant solution to fix it as far as I know. You could do retry or set a keepalive_timeout value to the upstream connection pool in nginx to avoid the problem.

referencing:

Apache HttpClient Interim Error: NoHttpResponseException

http://tengine.taobao.org/document/http_upstream_keepalive_timeout.html

Answer 6

I was getting the same nginx error and also there was no information in uwsgi log. The problem was that in some cases the application was not consuming the whole request body as advised in http://uwsgi-docs.readthedocs.org/en/latest/ThingsToKnow.html:

If an HTTP request has a body (like a POST request generated by a form), you have to read (consume) it in your application. If you do not do this, the communication socket with your webserver may be clobbered. If you are lazy you can use the post-buffering option that will automatically read data for you. For Rack applications this is automatically enabled.

Of course, this is not a problem in your case, but it may be useful for others who are getting the same nginx error.