Robustly call a flaky API: proper error handling with Net::HTTP

Question

I hacked this together as a seemingly robust way to call a flaky webservice that was giving timeouts and the occasional name resolution or socket error or whatever. I thought I\

Answer 1

When attempting to handle error conditions, be aware that by default Net::HTTP will AUTOMATICALLY RETRY for some HTTP verbs. The source for net/http.rb#transport_request (Ruby v2.5.0) includes:

    if count < max_retries && IDEMPOTENT_METHODS_.include?(req.method)
      count += 1
      @socket.close if @socket
      D "Conn close because of error #{exception}, and retry"
      retry
    end

so unless precautions are taken, Net::HTTP code will call the service 2x for all HTTP verbs included in IDEMPOTENT_METHODS_ before returning an error.

For Ruby < 2.5, the best thing I've come up with in a Rails app is to add something like:

Net::HTTP::IDEMPOTENT_METHODS_.delete_if { true }

This works around the problem by making IDEMPOTENT_METHODS_ empty so the #include? on req.method will always fail.

For Ruby >= 2.5, Net::HTTP has a #max_retries= method that should be set to 0 (unless of course retries are desired).

More background and history may be found here: Ruby Users: Be Wary of Net::HTTP