Python check if website exists

Question

I wanted to check if a certain website exists, this is what I\'m doing:

user_agent = \'Mozilla/20.0.1 (compatible; MSIE 5.5; Windows NT)\'
headers = { \'User         


        

Answer 1

Try this one::

import urllib2  
website='https://www.allyourmusic.com'  
try:  
    response = urllib2.urlopen(website)  
    if response.code==200:  
        print("site exists!")  
    else:  
        print("site doesn't exists!")  
except urllib2.HTTPError, e:  
    print(e.code)  
except urllib2.URLError, e:  
    print(e.args)  

Answer 2

It's better to check that status code is < 400, like it was done here. Here is what do status codes mean (taken from wikipedia):

• 1xx - informational
• 2xx - success
• 3xx - redirection
• 4xx - client error
• 5xx - server error

If you want to check if page exists and don't want to download the whole page, you should use Head Request:

import httplib2
h = httplib2.Http()
resp = h.request("http://www.google.com", 'HEAD')
assert int(resp[0]['status']) < 400

taken from this answer.

If you want to download the whole page, just make a normal request and check the status code. Example using requests:

import requests

response = requests.get('http://google.com')
assert response.status_code < 400

See also similar topics:

• Python script to see if a web page exists without downloading the whole page?
• Checking whether a link is dead or not using Python without downloading the webpage
• How do you send a HEAD HTTP request in Python 2?
• Making HTTP HEAD request with urllib2 from Python 2

Hope that helps.