python mechanize forms() err

Question

I'm using Python 2.7.6 and mechanize 0.2.5 and I want to log in to 'dining.ut.ac.ir' (I have the username and password)- but when I try to run the below script to get the forms list:

import mechanize
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [('User-agent', 'Firefox')]    
br.open("http://dining.ut.ac.ir/")
br.forms()

I get this error:

Traceback (most recent call last):
  File "script.py", line 8, in <module>
    br.forms()
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_mechanize.py", line 420, in forms
    return self._factory.forms()
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_html.py", line 557, in forms
    self._forms_factory.forms())
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_html.py", line 237, in forms
    _urlunparse=_rfc3986.urlunsplit,
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_form.py", line 844, in ParseResponseEx
    _urlunparse=_urlunparse,
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_form.py", line 981, in _ParseFileEx
    fp.feed(data)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_form.py", line 758, in feed
    _sgmllib_copy.SGMLParser.feed(self, data)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_sgmllib_copy.py", line 110, in feed
    self.goahead(0)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_sgmllib_copy.py", line 144, in goahead
    k = self.parse_starttag(i)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_sgmllib_copy.py", line 302, in parse_starttag
    self.finish_starttag(tag, attrs)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_sgmllib_copy.py", line 347, in finish_starttag
    self.handle_starttag(tag, method, attrs)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_sgmllib_copy.py", line 387, in handle_starttag
    method(attrs)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_form.py", line 735, in do_option
    _AbstractFormParser._start_option(self, attrs)
  File "/home/arman/workspace/python/mechanize/venv/lib/python2.7/site-packages/mechanize/_form.py", line 480, in _start_option
    raise ParseError("OPTION outside of SELECT")
mechanize._form.ParseError: OPTION outside of SELECT

Why I get this error and how I can fix it?

Answer 1

The URL you are trying to open is GZipped (check it using this link), So you have to append Accept-Encoding header for gzip to your Browser:

import mechanize
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [('User-agent', 'Firefox')]
br.addheaders.append( ['Accept-Encoding','gzip'] )
br.open("http://dining.ut.ac.ir/")
br.forms()

Answer 2

I had faced this same issue earlier and this line of code fixed my problem:

br = mechanize.Browser(factory=mechanize.RobustFactory())

So, try with this:

import mechanize
br = mechanize.Browser(factory=mechanize.RobustFactory())
br.set_handle_robots(False)
br.addheaders = [('User-agent', 'Firefox')]    
br.open("http://dining.ut.ac.ir/")
br.forms