Process AJAX request in Htmlunit

Question

I have a program written to scrape the source code from a webpage after a button is clicked. I am unable to scrape the right page because I believe an AJAX request is being

Answer 1

I would try the solution of setting

webClient.setAjaxController(new NicelyResynchronizingAjaxController());

this would cause all ajax calls to be synchronous.

Alternatively, did you try in your solution to call to "webClient.waitForBackgroundJavaScript(10000)" after tou got the page?

Something like this:

final HtmlPage page2 = button.click();
webClient.waitForBackgroundJavaScript(10000)
String originalHtml = page2.asXml();
return originalHtml;

Please use also htmlunit 2.13

Answer 2

For me it helped to use htmlunit 2.15 with NicelyResynchronizingAjaxController, and also

webClient.getOptions().setThrowExceptionOnScriptError(false);

My full setup is

    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());