Puppeteer

finding recaptcha callback

ⅰ亾dé卋堺 提交于 2021-01-28 05:22:36
问题 im trying to get data from this page https://ahrefs.com/backlink-checker its basically a website to check a domain rank and other status , when u enter a domain and click the check Check backlinks button it shows a google recaptcha im using a captcha service to bypass this , problem is this site uses a callback on the captcha completion , when i recive the token from my api and put it in the #g-recaptcha-response i have to call the callback to move on there is no submit button i used to find

Get all values from localStorage using Puppeteer

萝らか妹 提交于 2021-01-28 04:57:32
问题 Is it possible to get all the values from localStorage using Puppeteer? including values from third-party domains (with the assumption that I don't know all the third-party domains). I am looking for something similar with this, which gets all the cookies from the browser (but for localStorage ). export const getCookies = async page => { const { cookies } = await page._client.send("Network.getAllCookies", {}); return cookies; }; 回答1: However, if we suppose that localStorage origins = frames,

Puppeteer doesn't work at VPS (DigitalOcean)

大城市里の小女人 提交于 2021-01-27 22:53:53
问题 I'm in a droplet at DigitalOcean and i'm getting this error. (node:5549) UnhandledPromiseRejectionWarning: TimeoutError: Navigation Timeout Exceeded: 300000ms exceeded at Promise.then (/var/www/screenshot/node_modules/puppeteer/lib/NavigatorWatcher.js:94: at <anonymous> The url that I'm trying to screenshot is https://www.pontofrio.com.br/ I added an user agent to bypass the protection against headless request. It worked in my local machine but when I run at my VPS it get the error, even if I

Puppeteer not behaving like in Developer Console

孤人 提交于 2021-01-27 21:57:19
问题 I am trying to extract using Puppeteer the title of this page: https://www.nordstrom.com/s/zella-high-waist-studio-pocket-7-8-leggings/5460106 I have the below code, (async () => { const browser = await puppet.launch({ headless: true }); const page = await browser.newPage(); await page.goto(req.params[0]); //this is the url title = await page.evaluate(() => { Array.from(document.querySelectorAll("meta")).filter(function ( el ) { return ( (el.attributes.name !== null && el.attributes.name !==

How to click a button on a website using Puppeteer without any class, id ,… assigned to it?

对着背影说爱祢 提交于 2021-01-27 14:31:03
问题 So I want to click on a button on a website. The button has no id, class,... So I should find a way to click the button with the name that's on it. In this example I should click by the name "Supreme®/The North Face® Leather Shoulder Bag" This is my code in Node.js const puppeteer = require('puppeteer'); let scrape = async () => { const browser = await puppeteer.launch({headless: false}); const page = await browser.newPage(); await page.goto('https://www.supremenewyork.com/shop/all/bags');

How to run lighthouse for the homepage after login from puppeteer

扶醉桌前 提交于 2021-01-27 07:02:12
问题 I added two npm "@lhci/cli" and puppeteer.After that I added two config file lighthouserc.js : config details are: module.exports = { ci: { upload: { target: 'temporary-public-storage' }, collect: { puppeteerScript: 'puppeteer-script.js', chromePath: puppeteer.executablePath(), url: ["https://myWebsite.com/abc"], headful: true, numberOfRuns: 1, disableStorageReset: true, setting: { disableStorageReset: true }, puppeteerLaunchOptions: { slowMo: 20, headless: false, disableStorageReset: true }

How to run lighthouse for the homepage after login from puppeteer

☆樱花仙子☆ 提交于 2021-01-27 07:00:27
问题 I added two npm "@lhci/cli" and puppeteer.After that I added two config file lighthouserc.js : config details are: module.exports = { ci: { upload: { target: 'temporary-public-storage' }, collect: { puppeteerScript: 'puppeteer-script.js', chromePath: puppeteer.executablePath(), url: ["https://myWebsite.com/abc"], headful: true, numberOfRuns: 1, disableStorageReset: true, setting: { disableStorageReset: true }, puppeteerLaunchOptions: { slowMo: 20, headless: false, disableStorageReset: true }

Why Puppeteer needs --no-sandbox to launch Chrome in Cloud Functions

╄→гoц情女王★ 提交于 2021-01-27 04:23:13
问题 When I run Puppeteer on Cloud Functions with Node 8, I get this error. Error: Failed to launch chrome! [1205/063825.588245:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180. Launching Puppeteer with browser = await puppeteer.launch({ args: ['--no-sandbox'], headless: true }) solves the issue. I have searched on the internet and could not find any reason why is this necessary? Why does Headless Chrome run as root in the

Puppeteer does not activate button click, despite selecting button

折月煮酒 提交于 2021-01-25 07:00:24
问题 I'm trying to automate a sign in to a simple website that a scammer sent my friend. I can use puppeteer to fill in the text inputs but when I try to use it to click the button, all it does is activate the button color change (that happens when the mouse hovers over the button). I also tried clicking enter while focusing on the input fields, but that doesn't seem to work. When I use document.buttonNode.click() in the console, it worked, but I can't seem to emulate that with puppeteer I also

Puppeteer does not activate button click, despite selecting button

北战南征 提交于 2021-01-25 07:00:23
问题 I'm trying to automate a sign in to a simple website that a scammer sent my friend. I can use puppeteer to fill in the text inputs but when I try to use it to click the button, all it does is activate the button color change (that happens when the mouse hovers over the button). I also tried clicking enter while focusing on the input fields, but that doesn't seem to work. When I use document.buttonNode.click() in the console, it worked, but I can't seem to emulate that with puppeteer I also