Puppeteer evaluate function

问题

I'm new to pupetteer and I'm trying to understand how it's actually working through some examples:

So basically what I'm trying to do in this example is to extract number of views of a Youtube video. I've written a js line on the Chrome console that let me extract this information:

document.querySelector('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').innerText

Which worked well. However when I did the same with my pupetteer code he doesn't recognize the element I queried.

const puppeteer = require('puppeteer')

const getData = async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()

  await page.goto('https://www.youtube.com/watch?v=T5GSLc-i5Xo')
  
  await page.waitFor(1000)

  const result = await page.evaluate(() => {
    let views = document.querySelector('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').innerText
    return {views}
  })

  browser.close()
  return result
}

getData().then(value => {
  console.log(value)
})

I finally did it using ytInitialData object. However I'd like to understand the reason why my first code didn't work.

Thanks

回答1:

It seems that wait for 1000 is not enough.

Try your solution with https://try-puppeteer.appspot.com/ and you will see.

However if you try the following solution, you will get the correct result

const browser = await puppeteer.launch();

const page = await browser.newPage();
await page.goto('https://www.youtube.com/watch?v=T5GSLc-i5Xo');

await page.waitForSelector('span.view-count');
const views = await page.evaluate(() => document.querySelector('span.view-count').textContent);
console.log('Number of views: ' + views);

await browser.close();

回答2:

Do not use hand made timeout to wait a page to load, unless you are testing whether the page can only in that amount of time. Differently from selenium where sometimes you do not have a choice other than using a timeout, with puppeteer you should always find some await function you can use instead of guessing a "good" timeout. As answered by Milan Hlinák, look into the page HTML code and figure out some HTML tag you can wait on, instead of using a timeout. Usually, wait for the HTML element(s) you test require in order to work properly. On you case, the span.view-count, as already answered by Milan Hlinák:

await page.waitForSelector('span.view-count');

来源：https://stackoverflow.com/questions/53904333/puppeteer-evaluate-function

标签

javascript

node.js

Puppeteer