Python urllib,urllib2 fill form

Question

I want to fill a HTML form with urllib2 and urllib.

import urllib
import urllib2

url = 'site.com/registration.php'
values = {'password' : 'password',
          'username': 'username'
          }

data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()

But on the end of the form is a button(input type='submit'). If you don't click the button you can't send the data what you wrote in the input(type text)

How can I click the button with urllib and urllib2?

Answer 1

This is really more of something you would do with Selenium or similar. selenium.webdriver.common.action_chains.ActionChains.click, perhaps.

Answer 2

IF you look at your forms action attribute you could figure out what uri your form is submitting to. You can then make a post request to that uri. This can be done like :

import urllib
import urllib2

url = 'http://www.someserver.com/cgi-bin/register.cgi'
values = {'name' : 'Michael Foord',
          'location' : 'Northampton',
          'language' : 'Python' }

data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()

https://docs.python.org/2/howto/urllib2.html

You could also use the requests library which makes it a lot easier. You can read about it here

Another thing you might need to factor in is the CSRF(Cross Site Request Forgery) token which is embedded in forms. You will have to some how acquire it and pass it in with your post.