Submitting a post request to an aspx page

Question

I have an ASPX page at https://searchlight.cluen.com/E5/CandidateSearch.aspx with a form on it, that I\'d like to submit and parse for information.

Using Python\'s

Answer 1

I tried mechanize and urllib2, and mechanize handles cookies better. I can submit the form simply by specifying with mechanize:

    browser= mechanize.Browser()
    browser.select_form(form_name)
    browser.set_value("Page$Next", name="pagenumber")     

It was not necessary to replicate the post request manually, and mechanize in this case was able to handle a form that relies on javascript.

Answer 2

ASP.Net uses a security feature that protects against tampering with the ViewState by embedding specific information in it.

More than likely, the server is rejecting your request because the ViewState is being treated as though it were tampered with. I can't say this with absolute certainty, but ASP.Net has several security features that are built in to the framework that may be preventing a direct post.

If session is involved at all, then you will also need to take that into account. To simulate what the browser is doing you will need to perform the following steps:

1. Request the page.
2. Save the collection of cookies to a variable.
3. Extract the ViewState to a variable.
4. Post with the appropriate form values, passing both the saved cookies and ViewState information along with the request.

A lot of work I know, but not too awfully difficult. Again, this may not be the sole source of your problems, but it is worth reading up on in order to start troubleshooting.