How do I authenticate a urllib2 script in order to access HTTPS web services from a Django site?

Question

everybody. I\'m working on a django/mod_wsgi/apache2 website that serves sensitive information using https for all requests and responses. All views are written to redirect

Answer 1

Please excuse my answering my own question, but - for the record this seems to have solved it:

It turns out I needed to set the HTTP Referer header to the login page url in the request where I post the login information.

req.add_header( 'Referer', login_url )

The reason is explained on the Django CSRF documentation - specifically, step 4.

Due to our somewhat peculiar server setup where we use HTTPS on the production side and DEBUG=False, I wasn't seeing the csrf_failure reason for failure (in this case: 'Referer checking failed - no referer') that is normally output in the DEBUG info. I ended up printing that failure reason to the Apache error_log and STFW'd on it. That lead me to code.djangoproject/.../csrf.py and the Referer header fix.

Answer 2

This works on my django setup on https which is inspired by yours. I'm starting to think that the problem is outside this code... Is the server saying anything? I might very well be looking into apache.

I'm using the following code from my local machine to my server using ssl on nginx, so apache might be the place to look. I suppose one way to narrow it down is to try your script on my login page :) Shoot me an email!

import urllib
import urllib2
import contextlib


def login(login_url, username, password):
    """
    Login to site
    """
    cookies = urllib2.HTTPCookieProcessor()
    opener = urllib2.build_opener(cookies)
    urllib2.install_opener(opener)

    opener.open(login_url)

    try:
        token = [x.value for x in cookies.cookiejar if x.name == 'csrftoken'][0]
    except IndexError:
        return False, "no csrftoken"

    params = dict(username=username, password=password, \
        this_is_the_login_form=True,
        csrfmiddlewaretoken=token,
         )
    encoded_params = urllib.urlencode(params)

    with contextlib.closing(opener.open(login_url, encoded_params)) as f:
        html = f.read()

        print html
        # we're in.

---