Why is Python 3 http.client so much faster than python-requests?

Question

I was testing different Python HTTP libraries today and I realized that http.client library seems to perform much much faster than requests.

To test it you can run foll

Answer 1

copy-pasting response from @Lukasa posted here:

The reason Requests is slower is because it does substantially more than httplib. httplib can be thought of as the bottom layer of the stack: it does the low-level wrangling of sockets. Requests is two layers further up, and adds things like cookies, connection pooling, additional settings, and kinds of other fun things. This is necessarily going to slow things down. We simply have to compute a lot more than httplib does.

You can see this by looking at cProfile results for Requests: there's just way more result than there is for httplib. This is always to be expected with high-level libraries: they add more overhead because they have to do a lot more work.

While we can look at targetted performance improvements, the sheer height of the call stack in all cases is going to hurt our performance markedly. That means that the complaint that "requests is slower than httplib" is always going to be true: it's like complaining that "requests is slower than sending carefully crafted raw bytes down sockets." That's true, and it'll always be true: there's nothing we can do about that.