mechanize open Url python

Question

I am trying to open a URL using mechanize in python. The code executes with no errors, but nothing actually happens. What am I missing? Also, is there a way to set the browser? This is python 2.7.

import mechanize
url='http://www.google.com/'
op = mechanize.Browser() # use mecahnize's browser
op.set_handle_robots(False) #tell the webpage you're not a robot
op.open(url)

Answer 1

mechanize doesn't use real browsers - it is a tool for programmatic web-browsing.

For example, print out the page title after opening the url:

>>> import mechanize
>>> url='http://www.google.com/'
>>> op = mechanize.Browser() 
>>> op.set_handle_robots(False) 
>>> op.open(url)
<response_seek_wrapper at 0x10247ebd8 whose wrapped object = <closeable_response at 0x102479a70 whose fp = <socket._fileobject object at 0x101903950>>>
>>> op.title()
'Google'

---

Here's a follow up, how you can submit the Google search form:

import mechanize


url='http://www.google.com/'
op = mechanize.Browser()

op.set_handle_equiv(True)
op.set_handle_gzip(True)
op.set_handle_redirect(True)
op.set_handle_referer(True)
op.set_handle_robots(False)

# pretend you are a real browser
op.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]

op.open(url)

op.select_form(nr=1)
op.form['q'] = 'Does mechanize use a real browser?'
op.submit()

print op.geturl()

Prints:

http://www.google.com/search?hl=en&source=hp&q=Does+mechanize+use+a+real+browser%3F&btnG=Google+Search&gbv=1

Answer 2

If your goal is to open a page in an actual web browser, I suggest instead of using mechanize that you use the webbrowser module included by default in Python 2.7. The simplest use of this module can be demonstrated by the command

>>> import antigravity

which opens up http://xkcd.com/353/ in your browser. The code for this joke module is

import webbrowser

webbrowser.open("http://xkcd.com/353/")

There are many options for customizing the behavior including which browser opens available. You can read up on these in the webbrowser docs.