got Validation of viewstate MAC failed when sending post request from google app engine via url fetch service

Question

I have a task to fetch html from a website, before I go to that page I need to log in.

I use a low-level api url fetch service. Here is my code test code:

Answer 1

It looks like you are doing a POST to an aspx page.

When an aspx page receives a POST request it expects some hidden inputs which have an encoded ViewState present - if you browse to the page in question and "View Source" you should see some fields just inside the <form /> tag that look something like this:

<input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value="" />
<input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value="" />
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="xxxxxxxxx" />

Because you are submitting a POST request without these values present, it's having trouble decoding and validating them (which is what that error means - it can also crop up for other reasons in other scenarios).

There are a couple of possible solutions to this:

1 - If you have access to the code for the site, and the login page doesn't require ViewState, you could try switching it off at the page level within the @Page directive:

<%@ Page ViewStateMode="Disabled" .... %>

2 - You could do a double-request - do a GET request on the login page to retrieve the values for any missing hidden fields - use those values and include them in your POST

EDIT Ah yes, from your comment I can see that you're including the hidden form fields already - apologies!

In which case, another possibility is that the login page is on a load balanced environment. Each server in that environment will have a different MachineKey value (which is used to encode/decode the ViewState). You may be reading from one and posting to the other. Some LBs inject ArrowPoint cookies into the response to ensure that you "stick" to the same server between requests.

I can see you're already including a cookie in your POST, but I can't see where it's defined. Is it from the first GET request, or is it a custom cookie? If you haven't tried it already, maybe try using the cookie from the original GET where you're retrieving the login page HTML? Other than that, I'm out of ideas - sorry!

Answer 2

Commonly, when you're trying to emulate a postBack on the asp.net, you need to POST:

• preserved from the first request cookies to act on the same session
• data fields (login, password)
• hidden ones from the first page: __VIEWSTATE, __VIEWSTATEENCRYPTED (even if it's empty!), __EVENTVALIDATION
• if you sending some action items, maybe you need to include also hidden fields __EVENTTARGET and __EVENTARGUMENT