Bandwidth throttling in Python

Question

What libraries out there let you control the download speed of network requests (http in particular). I don\'t see anything built-in in urllib2 (nor in (Py)Qt which I intend on

Answer 1

urllib2 doesn't offer a way to do this, so you'd have to extend some of the classes it uses and implement rate limiting yourself. You might want to look at this question. If you decide to write a limiter, read up on the token bucket and leaky bucket algorithms.

Alternatively, you could use pycurl along with the CURLOPTMAXRECVSPEEDLARGE option.

EDIT: The urlgrabber package appears to support throttling as well, and is probably easier to understand than pycurl.

If you prefer to program using an event loop model, there's the Twisted approach, which has already been mentioned in another answer.

Answer 2

Of course twisted can. You want twisted.protocols.policies.ThrottlingFactory. Just wrap your existing factory in it before you pass it to whatever wants a factory.