httr

问题 When a connection is created with open="r" it allows for line-by-line reading, which is useful for batch processing large data streams. For example this script parses a sizable gzipped JSON HTTP stream by reading 100 lines at a time. However unfortunately R does not support SSL: > readLines(url("https://api.github.com/repos/jeroenooms/opencpu")) Error in readLines(url("https://api.github.com/repos/jeroenooms/opencpu")) : cannot open the connection: unsupported URL scheme The RCurl and httr

问题 I'm trying to use Rvest to scrape a table off of an internal webpage here at $JOB. I've used the methods listed here to get the xpath, etc. My code is pretty simple: library(httr) library(rvest) un = "username"; pw = "password" thexpath <- "//*[@id="theFormOnThePage"]/fieldset/table" url1 <- "https://biglonghairyURL.do?blah=yadda" stuff1 <- read_html(url1, authenticate(un, pw)) This gets me an error of: "Peer certificate cannot be authenticated with given CA certificates." Leaving aside the

问题 I'm trying to use the Mashape platform to access to different APIs with R (e.g. epguides or pipl). I could use directly the original APIs, but getting used to Mashape seems to be a good investment since it provides a unified access to a whole lot of other APIs. Two concerns however: Mashape doesn't provide any R tutorial. I tried the httr package to query Mashape but no success until there. How to query Mashape with R ? ; As far as I tested, most APIs endpoint on Mashape seem not to respond

问题 I would like to get response headers from GET or POST. My example is: library(httr) library(RCurl) url<-'http://www.omegahat.org/RCurl/philosophy.html' doc<-GET(url) names(doc) [1] "url" "handle" "status_code" "headers" "cookies" "content" "times" "config" but there is no response headers, only request headers. Result shoud be something like this: Connection:Keep-Alive Date:Mon, 11 Feb 2013 20:21:56 GMT ETag:"126a001-e33d-4c12cf2702440" Keep-Alive:timeout=15, max=100 Server:Apache/2.2.14

问题 I am trying to login with my credentials to a .NET site but unable to get it working. My code is inspired from the below thread How to login and then download a file from aspx web pages with R library(RCurl) curl = getCurlHandle() curlSetOpt(cookiejar = 'cookies.txt', followlocation = TRUE, autoreferer = TRUE, curl = curl) html <- getURL('http://www.aceanalyser.com/Login.aspx', curl = curl) viewstate <- as.character(sub('.*id="__VIEWSTATE" value="([0-9a-zA-Z+/=]*).*', '\\1', html))

问题 I've got a manual process where I'm uploading 5-6 GB file to a web server via curl: curl -X POST --data-binary @myfile.csv http://myserver::port/path/to/api This process works fine, but I'd love to automate it using R. The problem is, I either don't know what I'm doing, or the R libraries for curl don't know how to handle files bigger than ~2GB: library(RCurl) postForm( "http://myserver::port/path/to/api", file = fileUpload( filename = path.expand("myfile.csv"), contentType = "text/csv" ),

问题 I am wanting to fill in a web form and submit my query and download the resulting data. Some of the fields have the option of a drop-down menu or typing in a search query, sections can also be left blank (if all sections are left blank the entire database is downloaded), hitting the "search and download" button should instigate the downloading of a file. Here is what I have tried (selecting all records for species "Salmo salar") based on this question. I used my browser (Opera) "Developer

问题 I am trying to interrogate this site to get the list of offers. The problem is that we need to fill 2 forms (2 POST queries) before receiving the final result. This what I have done so far: First I am sending the first POST after setting the cookies: library(httr) set_cookies(.cookies = c(a = "1", b = "2")) first_url <- "https://compare.switchon.vic.gov.au/submit" body <- list(energy_category="electricity", location="home", "location-home"="shift", "retailer-company"="", postcode="3000",

问题 note: ipums international and ipums usa probably use the same system. ipums usa allows quicker signup. if you would like to test out your code, try https://usa.ipums.org/usa-action/users/request_access to sign up! i am trying to programmatically download a file from https://international.ipums.org/ with the R language and httr. i need to use httr and not RCurl because i need to post-authentication download large files not into RAM but directly to disk. this is currently only possible with

问题 I'm trying to fetch a JSON array from my server using the HTTP POST method in R. I've tried using both the POST function from httr and the getURL function from RCurl but both return errors. cafile <- system.file("CurlSSL", "cacert.pem", package = "RCurl") url <- "https://example.com/query/getData.php" POST(url,body=NULL) POST(url,body=NULL,config(cainfo=cafile)) getURL(url) getURL(url,cainfo=cafile) The error given by the POST function is (for both calls): Error in curl::curl_fetch_memory(url