Random “[Errno -2] Name or service not known” errors

问题

I am populating a local database using a third party service. I have a list of urls (around 500). I am calling each url in a loop, and updating my database with the returned data. The code flow looks like this:

for url in urllist:
    req = urllib.urlopen(url)
    data = json.loads(req.read())
    req.close()

    #update the db using data here

Whenever I run this piece of code, the script fails at random points with the error message "Name or service not known". This doesn't have anything with the urls because the script fails at random points (i.e. at 50th iteration in one run, and at 60th iteration in another)

What could be the reason for this?

回答1:

if you use a bad proxy or there are network problems you can try this:

for url in urllist:
    retry = 0
    while True: # retry request
        try:
            req = urllib.urlopen(url)
            resp_data = req.read() # in call read() network still processing
        except Exception as e: # TODO need more detailed handling
            if retry > 3: # 3 this is serious problem. exit
                raise e
            retry += 1 # retry
        else:
            data = json.loads()
            req.close() # not needed
            break

来源：https://stackoverflow.com/questions/20264953/random-errno-2-name-or-service-not-known-errors