CasperJS, Page Stuck on Loading Screen

问题

I'm trying to use casperjs to grab a comment off of this page:

http://www.regulations.gov/#!documentDetail;D=APHIS-2013-0013-0083

but when I navigate casper to it, it gets stuck on the 'loading' page. This is the image that I took using casper.capture();

My full code:

var x = require('casper').selectXPath;

casper.userAgent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)');

casper.start('http://www.regulations.gov/#!documentDetail;D=APHIS-2013-0013-0083');

casper.then(function () {
casper.capture('ourImage1.png');
console.log('Copying comment from Regulations...');
casper.wait(3000, function() {
    var text = casper.fetchText(x('/html/body/div[3]/div[2]/div[2]/div[3]/div/table/tbody/tr/td[1]/div/div[3]/div[1]/div/div[2]'));
    console.log(text);
    casper.wait(3000, function() {
        casper.capture('ourImage2.png');
    });
});
});

casper.run();

I even gave it up to 30 seconds to load with casper.wait(), but nothing seems to work. It just gets stuck with the loading page.

回答1:

This works perfectly fine if you change this (horrible) userAgent or remove the line altogether:

casper.userAgent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)');

It may be possible that the site provides special IE 6 optimized version that breaks phantomjs parsing. When I added the following code in front of the test, it displayed multiple errors that went away without the userAgent.

casper.on("remote.message", function(msg){
    this.echo("remote.msg: " + msg);
});

casper.on("resource.error", function(resourceError){
    this.echo("res.err: " + JSON.stringify(resourceError));
});

来源：https://stackoverflow.com/questions/24292882/casperjs-page-stuck-on-loading-screen

标签

javascript

jquery

web-scraping

casperjs