发表新帖
发表新帖

Use PhantomJS to extract html and text

前端 未结
关注
4 710
眼角桃花
眼角桃花 2020-12-21 23:11

I try to extract all the text content of a page (because it doesn\'t work with Simpledomparser)

I try to modify this simple example from the manual
相关标签:
4条回答
  • 别那么骄傲
    2020-12-21 23:32

    This version of your script should return the entire contents of the page:

    var page = require('webpage').create();
page.settings.userAgent = 'SpecialAgent';
page.open('http://www.httpuseragent.org', function (status) {
    if (status !== 'success') {
        console.log('Unable to access network');
    } else {
        var ua = page.evaluate(function () {
            return document.getElementsByTagName('html')[0].outerHTML;
        });
        console.log(ua);
    }
    phantom.exit();
});
    0 讨论(0)
  • 甜味超标
    2020-12-21 23:32

    Having encountered this question while trying to solve a similar problem, I ended up adapting a solution from this question like so:

    var fs = require('fs');
var file_h = fs.open('header.html', 'r');
var line = file_h.readLine();
var header = "";

while(!file_h.atEnd()) {

    line = file_h.readLine(); 
    header += line;

}
console.log(header);

file_h.close();
phantom.exit();

    This gave me a string with the read-in HTML file that was sufficient for my purposes, and hopefully may help others who came across this.

    The question seemed ambiguous (was it the entire content of the file required, or just the "text" aka Strings?) so this is one possible solution.

    0 讨论(0)
  • 一个人的身影
    2020-12-21 23:39

    To extract the text content of the page, you can try thisreturn document.body.textContent; but I'm not sure the result will be usable.

    0 讨论(0)
  • 刺人心
    2020-12-21 23:42

    There are multiple ways to retrieve the page content as a string:

    • page.content gives the complete source including the markup (<html>) and doctype (<!DOCTYPE html>),

    • document.documentElement.outerHTML (via page.evaluate) gives the complete source including the markup (<html>), but without doctype,

    • document.documentElement.textContent (via page.evaluate) gives the cumulative text content of the complete document including inline CSS & JavaScript, but without markup,

    • document.documentElement.innerText (via page.evaluate) gives the cumulative text content of the complete document excluding inline CSS & JavaScript and without markup.

    document.documentElement can be exchanged by an element or query of your choice.

    0 讨论(0)
 看不清?
提交回复
热议问题