Cannot get link to be clicked and switch to next page using PhantomJS

Question

I am having an issue getting phantomJS to click the login button on a website.

I can see in my second screenshot that it is trying to select the login button, but I

Answer 1

page.sendEvent() is a synchronous function that finishes as soon as its action is done. The next call (page.render()) is executed even before the request which was triggered by the click is answered.

1. setTimeout

JavaScript provides two functions to wait a static amount of time: setTimeout and setInterval:

page.sendEvent('click', a.offsetLeft, a.offsetTop);

setTimeout(function(){
    page.render('TC0002.png');
    phantom.exit();
}, 5000);

(don't forget to remove the other phantom.exit() since you don't want to exit too early)

Of course the problem is now that on one hand the page still might not be ready after 5 seconds or on the other hand the page was loaded extremely fast and just sits there doing nothing.

2. waitFor

A better approach would be to use the waitFor() function that is provided in the examples folder of PhantomJS. You can wait for a specific condition of the page like the existence of a specific element:

page.sendEvent('click', a.offsetLeft, a.offsetTop);

waitFor(function _testFx(){
    return page.evaluate(function(){
        return !!document.querySelector("#someID");
    });
}, function _done(){
    page.render('TC0002.png');
    phantom.exit();
}, 10000);

3. page.onLoadFinished

Another approach would be to listen to the page.onLoadFinished event which will be called when the next page is loaded, but you should register to it before you click:

page.onLoadFinished = function(){
    page.render('TC0002.png');
    phantom.exit();
};

page.sendEvent('click', a.offsetLeft, a.offsetTop);

4. page.onPageCreated

Whenever a new window/tab would be opened in a desktop browser, the page.onPageCreated would be triggered in PhantomJS. It provides a reference to the newly created page, because the previous page is not overwritten.

page.onPageCreated = function(newPage){
    newPage.render('TC0002.png');
    newPage.close();

    phantom.exit();
};

page.sendEvent('click', a.offsetLeft, a.offsetTop);

In all the other cases, the page instance is overwritten by the new page.

5. "Full" page load

That might still not be sufficient, because PhantomJS doesn't specify what it means when a page is loaded and the JavaScript of the page may still make further requests to build up the page. This Q&A has some good suggestions to wait for a "full" page load: phantomjs not waiting for “full” page load