Python - resume web session from urllib2 after manual browser login

Question

Say, I browse to a website (on intranet too) that require a login to access the contents. I will fill in the required fields... e.g. username, password and any captcha, etc.

Answer 1

You want to use the cookielib module.

http://docs.python.org/library/cookielib.html

You can log on using your browser, then export the cookies into a Netscape-style cookie.txt file. Then from python you'll be able to load this and fetch the resource you require. The cookie will be good until the website expires your session (often around 30 days).

import cookielib, urllib2
cj = cookielib.MozillaCookieJar()
cj.load('cookie.txt')
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/resource")

There are add-ons for Chrome and Firefox that will export the cookies in this format. For example:

https://chrome.google.com/webstore/detail/lopabhfecdfhgogdbojmaicoicjekelh

https://addons.mozilla.org/en-US/firefox/addon/export-cookies/