I get HTTP 500 error when use python urllib2.open

问题

The code is like. The url is my blog and I'd like to fetch and backup my blog posts. Things goes well with my other blog posts but return 500 error with this one.

usock = urlopen("http://xiaoshuaistudio.ycool.com/post.4606754.html")
htmlSource = usock.read()
usock.close()

Can you help me figure out how to debug a HTTP 500 error?

回答1:

A page content is loaded, and a 500 error is returned. But I myself can't read whether the page content is relevant.

To avoid error on this specific page or website you could do (WARNING this is not very pretty)

try:
  response = urlopen(url)
  content = response.read()
except HTTPError, e:
  if e.getcode() == 500:
    content = e.read()
  else:
    raise

回答2:

Take a look at the server logs on your web server. In general, error data will be safe on the server. It's not a good idea to dump error logs to users.

回答3:

I think there is something wrong with the ycool.com server, because I couldn't open this url with my web browser.

http://xiaoshuaistudio.ycool.com/post.4606754.html

That web server will always return status code 500 Internal Server Error for some url, which should be 200 OK.

If the urlopen function only check for the status code, it will return that error(Although the server send that page's content).

来源：https://stackoverflow.com/questions/8404938/i-get-http-500-error-when-use-python-urllib2-open