phantomjs

phantomjs学习

由 独自空忆成欢 提交于 2020-01-28 05:50:13
PhantomJS快速入门 　　本文简要介绍了PhantomJS的相关基础知识点，主要包括PhantomJS的介绍、下载与安装、HelloWorld程序、核心模块介绍等。由于鄙人才疏学浅，难免有疏漏之处，欢迎指正交流。 　　1、PhantomJS是什么？ 　　PhantomJS是一个基于webkit的JavaScript API。它使用QtWebKit作为它核心浏览器的功能，使用webkit来编译解释执行JavaScript代码。任何你可以在基于webkit浏览器做的事情，它都能做到。它不仅是个隐形的浏览器，提供了诸如CSS选择器、支持Web标准、DOM操作、JSON、HTML5、Canvas、SVG等，同时也提供了处理文件I/O的操作，从而使你可以向操作系统读写文件等。PhantomJS的用处可谓非常广泛，诸如网络监测、网页截屏、无需浏览器的 Web 测试、页面访问自动化等。 　　PhantomJS官方地址：http://phantomjs.org/。 　　PhantomJS官方API：http://phantomjs.org/api/。 　　PhantomJS官方示例：http://phantomjs.org/examples/。 　　PhantomJS GitHub：https://github.com/ariya/phantomjs/。 　　2、PhantomJS下载与安装

PhantomJS - Login to Twitch Through jQuery

由 自作多情 提交于 2020-01-25 11:04:35
问题 I am looking for a way to login to twitch through PhantomJS. The code I have now is not working when I look through "e.png" var page = require('webpage').create(); page.open('http://www.twitch.tv/login', function() { page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() { page.evaluate(function() { $("login_user_login").val("username"); $("user[password]").val("password"); $("button").click(); // click login button page.render("e.png"); // see if

Using angularJS function in phantomJS

由 不想你离开。 提交于 2020-01-25 07:40:08
问题 I searched on the net and I found nothing. I have a script angularJS which appear or disappear element from my html page. I use a phantomJS script to access to the html page and do some stuffs. But Can I use the $scope or the functions of my angular script in my phantom script ? I know that I can use jQuery but angular i don't know. 回答1: I finally found. A script PhantomJS use javascript so we just have to call angularJS element like in javascript: This is angular.element($("#someThing"))

How to Implement Selenium WebDriver with PhantomJS and Can we Use Sikuli with PhantomJS?

由 泪湿孤枕 提交于 2020-01-24 21:14:25
问题 I am trying to use following code for running my script in headless browser in PhantomJS. public void setUp() throws Exception{ Capabilities caps = new DesiredCapabilities(); ((DesiredCapabilities) caps).setJavascriptEnabled(true); ((DesiredCapabilities) caps).setCapability("takesScreenshot", true); ((DesiredCapabilities) caps).setCapability( PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY, "/Users/kamal.kumar/Desktop/phantomjs-2.1.1-macosx/bin/phantomjs.exe" ); // System

How to get html generated from javascript using phantomJS?

由 风格不统一 提交于 2020-01-24 01:24:08
问题 var page = require('webpage').create(); page.open(url, function (status) { if (status === 'fail') { console.log("fail"); } else { console.log(page.evaluate(function () { var t0 = document.body.innerHTML; return t0; })); } phantom.exit(); }); When I do this way, I can't get the html generated from js. 回答1: This is not the correct way to return anything from the page scope to the phantom's scope. You should consider using window.callPhantom method and page.onCallback event. 来源： https:/

Using casperjs and PHP to save data

由 两盒软妹~` 提交于 2020-01-23 04:24:07
问题 I've been looking for some kind of tutorial or example on how to do this, but either I'm not understanding that the solution is right under my nose, or no one has really put this out there. What I'm trying to do is use casperjs to automate a process where I create an account on a website. I will be providing a few different user names, and then I want to output a file at the end with the username that was used for registration along with the password. If I don't need to use PHP to do this,

How to reduce PhantomJS's CPU and memory usage?

由 点点圈 提交于 2020-01-22 18:51:28
问题 I'm using PhantomJS via Python's webdriver lib. It eats lots of RAM and CPU, and it's an issue because I'd like to run as many instances as it's possible. Some google'ing didn't give me anything helpful. So I'll ask directly: Does the size matter? If I set driver.set_window_size(1280, 1024), will it eat more memory than 1024x768? Is there any option in the source code which can be turned off without real issues and which lead to significant memory usage reduce? Yes I still need images and CSS

pyspider + python2.7

由 青春壹個敷衍的年華 提交于 2020-01-22 16:44:52
升级pip pip install --upgrade pip pip安装pyspider pip install pyspider 安装phantomjs： https://phantomjs.org/download.html wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 yum -y install bzip2 tar -jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /opt/ mv phantomjs-2.1.1-linux-x86_64/ phantomjs 建立软链接 ln -s /opt/phantomjs/bin/phantomjs /usr/bin/ 安装依赖 yum -y install fontconfig 启动验证 phantomjs pyspider启动报错 ValueError: Invalid configuration: - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead. 在安装包中找到pyspider的资源包

爬虫实战4：用selenium爬取淘宝美食

由 ε祈祈猫儿з 提交于 2020-01-22 16:38:38
方案1：一次性爬取全部淘宝美食信息 1. spider.py文件如下 1 __author__ = 'Administrator' 2 from selenium import webdriver 3 from selenium.webdriver.common.by import By 4 from selenium.webdriver.support.ui import WebDriverWait 5 from selenium.webdriver.support import expected_conditions as EC 6 import re 7 from pyquery import PyQuery as pq 8 from config import * 9 import pymongo 10 11 client = pymongo.MongoClient(MONGO_URL) 12 db = client[MONGO_DB] 13 14 browser = webdriver.Chrome() 15 """ 16 如果把Chrome修改为使用PhantomJS 17 1. 首先需要安装phantomJS 18 2. 自定义一些配置参数，这里不加载图片以及使用缓存 19 browser = webdriver.PhantomJS(service_args

centos7.6下pyspider + python2.7安装

由 故事扮演 提交于 2020-01-22 15:23:46
1.升级pip pip install --upgrade pip 2.pip安装pyspider pip install pyspider 3.安装phantomjs： https://phantomjs.org/download.html wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 yum -y install bzip2 tar -jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /opt/ mv phantomjs-2.1.1-linux-x86_64/ phantomjs 建立软链接 ln -s /opt/phantomjs/bin/phantomjs /usr/bin/ 安装依赖 yum -y install fontconfig 启动验证 phantomjs 4.pyspider启动报错 ValueError: Invalid configuration: - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead. 在安装包中找到pyspider的资源包
订阅 phantomjs