httplib CannotSendRequest error in WSGI

Question

I've used two different python oauth libraries with Django to authenticate with twitter. The setup is on apache with WSGI. When I restart the server everything works great for about 10 minutes and then the httplib seems to lock up (see the following error).

I'm running only 1 process and 1 thread of WSGI but that seems to make no difference.

I cannot figure out why it's locking up and giving this CannotSendRequest error. I've spent a lot of hours on this frustrating problem. Any hints/suggestions of what it could be would be greatly appreciated.

File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py", line 92, in get_response
  response = callback(request, *callback_args, **callback_kwargs)

File "mypath/auth/decorators.py", line 9, in decorated
  return f(*args, **kwargs)

File "mypath/auth/views.py", line 30, in login
  token = get_unauthorized_token()

File "/root/storm/eye/auth/utils.py", line 49, in get_unauthorized_token
  return oauth.OAuthToken.from_string(oauth_response(req))

File "mypath/auth/utils.py", line 41, in oauth_response
  connection().request(req.http_method, req.to_url())

File "/usr/lib/python2.5/httplib.py", line 866, in request
  self._send_request(method, url, body, headers)

File "/usr/lib/python2.5/httplib.py", line 883, in _send_request
  self.putrequest(method, url, **skips)

File "/usr/lib/python2.5/httplib.py", line 770, in putrequest
  raise CannotSendRequest()

CannotSendRequest

Answer 1

This exception is raised when you reuse httplib.HTTP object for new request while you havn't called its getresponse() method for previous request. Probably there was some other error before this one that left connection in broken state. The simplest reliable way to fix the problem is creating new connection for each request, not reusing it. Sure, it will be a bit slower, but I think it's not an issue having you are running application in single process and single thread.

Answer 2

Also check your Python version. I had a similar situation after updating to Py-2.7 from Py-2.6. In Py-2.6 everything worked without any problems. Py-2.7 httplib uses HTTP/1.1 by default which made the server did not send back the Connection: close option in the respond, therefore the connection handling was broken. In my case this worked with HTTP/1.0 though.