问题
Problem: I have searched several websites/blogs/etc to find a solution but did not get to what I was looking for. The problem in short, is that I would like to scrape a site - but to get to that site - I have to get past the login page.
What I did: I did manage to use urllib2 and httplib to open the page, but even after logging in (no errors being displayed) the redirection of the login page as shown in the browser does not happen. My code was not too different than what was displayed here: How to use Python to login to a webpage and retrieve cookies for later usage? ; except that I did not use Cookies.
What am I looking for? I am not entirely sure what fields I need to be looking for besides the "username" and "password" fields. What I would like for the script to do is 1) Successfully login to the .aspx site and display a message of some sort that the login was successful 2) Redirect to another page after logging in, in order for me to scrape the data off from the site. 3) How to gather any site's POST/GET fields so I know that I am passing/calling the right parameters?
Any assistance/help/advise would be much appreciated.
来源:https://stackoverflow.com/questions/10164322/python-problems-getting-past-the-login-page-of-an-aspx-site