Python urllib2: Reading content body even during HTTPError exception?

Question

I\'m using urllib2 to fetch a a page via HTTP. Sometimes the resource throws a HTTP error 400 (Bad Request) when my request contains an error. However, that response also cont

Answer 1

You can read the response message from the HTTPError exception.

Python3 example

import urllib.request

try:
    request = urllib.request.Request('http://httpstat.us/418', headers={'Accept': 'text/plain', 'User-Agent': ''})
    with urllib.request.urlopen(request) as page:
        print('success: ' + page.read().decode())
except urllib.error.HTTPError as httpError:
        error = httpError.read().decode()
        print('error: ' + error)

Answer 2

import urllib2
try:
    request = urllib2.Request('http://www.somesite.com')
    response = urllib2.urlopen(req)
except urllib2.HTTPError as e:
    error_message = e.read()
    print error_message

Answer 3

You can treat the error as a response.

http://www.voidspace.org.uk/python/articles/urllib2.shtml#httperror

When an error is raised the server responds by returning an HTTP error code and an error page. You can use the HTTPError instance as a response on the page returned. This means that as well as the code attribute, it also has read, geturl, and info, methods.