Python urllib vs httplib?

Question

When would someone use httplib and when urllib?

What are the differences?

I think I ready urllib uses httplib, I am planning to make an app that will need to make http request and so far I only used httplib.HTTPConnection in python for requests, and reading about urllib I see I can use that for request too, so whats the benefit of one or the other?

Answer 1

urllib (particularly urllib2) handles many things by default or has appropriate libs to do so. For example, urllib2 will follow redirects automatically and you can use cookiejar to handle login scripts. These are all things you'd have to code yourself if you were using httplib.

Answer 2

I would like to say something about urllib, urllib2, httplib and httplib2.

The main different between urllib* and httplib* is that:

httplib and httplib2 handles HTTP/HTTPs request and response directly and give you more space to do your own job.

urllib and urllib2 are build upon httplib, they are more abstract and powerful, but sometimes won't fulfill your specified need about some HTTP related operations.

And for httplib and httplib2, I'd say that they are both HTTP client library. However httplib2 is much more powerful and have much more features than httplib.

As for urllib and urllib2, quote from this link:

urllib and urllib2 are both Python modules that do URL request related stuff but offer different functionalities. Their two most significant differences are listed below:

• urllib2 can accept a Request object to set the headers for a URL request, urllib accepts only a URL. That means, you cannot masquerade your User Agent string etc.
• urllib provides the urlencode method which is used for the generation of GET query strings, urllib2 doesn't have such a function. This is one of the reasons why urllib is often used along with urllib2.

I would recommend my personal blog Httplib Httplib2 Urllib Urllib2-what’s the Difference.

Hope it helps:-)

Answer 3

urllib/urllib2 is built on top of httplib. It offers more features than writing to httplib directly.

however, httplib gives you finer control over the underlying connections.

Answer 4

If you're dealing solely with http/https and need access to HTTP specific stuff, use httplib.

For all other cases, use urllib2.

Answer 5

If you need high level stuff like Caching, Keep-Alive, Compression or Authentication, tryhttplib2

Answer 6

For those folks moving things up to Py3 (and for some reason cannot or have not refactored to use the awesome requests module), this is a good transition between versions:

try:
    import http.client as httplib
except ImportError:
    import httplib

Works in both Python version sets.