Pause before retry connection in Python

Question

I am trying to connect to a server. Sometimes I cannot reach the server and would like to pause for a few seconds before trying again. How would I implement the pause feature in Python. Here is what I have so far. Thank you.

   while True:
        try:
            response = urllib.request.urlopen(http)
        except URLError as e:
            continue
        break

I am using Python 3.2

Answer 1

This will block the thread for 2 seconds before continuing:

import time
time.sleep(2)

Answer 2

In case you want to run lots of these in parallel, it would be much more scalable to use an asynchronous networking framework such as Twisted, where "sleeping" doesn't mean blocking a valuable and expensive OS thread from doing some other useful work. Here's a rough sketch of how you can attempt as many requests in parallel as you wish (set to 100), with timeout (5 seconds), delay (2 seconds) and a configurable number of retries (here, 10).

from twisted.internet import defer, reactor
from twisted.web import client

# A semaphore lets you run up to `token` deferred operations in parallel
semaphore = defer.DeferredSemaphore(tokens=100)

def job(url, tries=1, d=None):
    if not d:
        d = defer.succeed(None)
    d.addCallback(lambda ignored: client.getPage(url, timeout=5))
    d.addCallback(doSomethingWithData)
    def retry(failure):
        if tries > 10:
            return failure # give up
        else:
            # try again in 2 seconds
            d = defer.succeed(None)
            reactor.callLater(2, job, url, tries=tries+1, d=d)
            return d
    d.addErrback(retry)
    return d

for url in manyURLs:
    semaphore.run(job, url)