Python authenticate and launch private page using webbrowser, urllib and CookieJar

Question

I want to login with cookiejar and and launch not the login page but a page that can only be seen after authenticated. I know mechanize does that but besides not working for me now, I rather do this without it. Now I have,

import urllib, urllib2, cookielib, webbrowser
from cookielib import CookieJar

username = 'my_username'
password = 'my_password'
url = 'my_login_page'

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'my_username' : username, 'my_password' : password})
opener.open(url, login_data)
page_to_launch = 'my_authenticated_url'
webbrowser.open(page_to_launch, new=1, autoraise=1)

I am either able to login and dump the authenticated page to stdout, or launch the login page without recognizing the cookie, but I am not able to launch the page I want to after logging in. Help appreciated.

Answer 1

You could use the selenium module to do this. It starts a browser (chrome, Firefox, IE, etc) with an extension loaded into it that allows you to control the browser.

Here's how you load cookies into it:

from selenium import webdriver
driver = webdriver.Firefox() # open the browser

# Go to the correct domain
driver.get("http://www.example.com")

# Now set the cookie. Here's one for the entire domain
# the cookie name here is 'key' and it's value is 'value'
driver.add_cookie({'name':'key', 'value':'value', 'path':'/'})
# additional keys that can be passed in are:
# 'domain' -> String,
# 'secure' -> Boolean,
# 'expiry' -> Milliseconds since the Epoch it should expire.

# finally we visit the hidden page
driver.get('http://www.example.com/secret_page.html')

Answer 2

Your cookies aren't making it to the browser.

webbrowser has no facilities for accepting the cookies stored in your CookieJar instance. It's simply a generic interface for launching a browser with a URL. You will either have to implement a CookieJar that can store cookies in your browser (which is almost certainly no small task) or use an alternative library that solves this problem for you.