Programmatically logging into a site

Question

It might sound silly; but can we programmatically login into a site such as Linkedin by passing our user credentials (userid and password)? I am not talking about using OAuth or

Answer 1

You can login to many sites this way, using scripting. I generally prefer Python with BeautifulSoup, but there are many other possibilities for languages and libraries.

The general idea is to locate the code for the form that has the username and password fields, fill them in, then post the form.

This works for me on Linux, with my LinkedIn username and password in my .netrc in the standard format:

#!/usr/bin/python
import sys, os, urllib, urllib2, cookielib, urlparse, netrc
from BeautifulSoup import BeautifulSoup
URL = 'https://www.linkedin.com/uas/login'
COOKIES = urllib2.HTTPCookieProcessor(cookielib.CookieJar())
HANDLER = getattr(urllib2, urlparse.urlsplit(URL).scheme.upper() + 'Handler')
DEBUGLEVEL = 0  # set to 1 to see unencrypted data
OPENER = urllib2.build_opener(COOKIES, HANDLER(debuglevel = DEBUGLEVEL))
def fetch(url, data = None):
 connection = OPENER.open(url, data)
 page = connection.read()
 connection.close()
 return page
def login():
 soup = BeautifulSoup(fetch(URL))
 form = soup.find('form')
 fields = form.findAll('input')
 auth_info = netrc.netrc().authenticators(urlparse.urlsplit(URL).netloc)
 formdata = dict([[field['name'], field['value']] for field in fields])
 formdata['session_key'], ignored, formdata['session_password'] = auth_info
 assert form['method'] == 'POST'
 posturl = urlparse.urljoin(URL, form['action'])
 print fetch(posturl, urllib.urlencode(formdata))
if __name__ == '__main__':
 login()