Copying cookie for httr

问题

I'm trying to access a site that sets a cookie on the basis of what type of visitor:

library(httr)
url <- "https://www.blackrock.com/ca/individual/en/products/product-list#categoryId=1&lvl2=overview"

Essentially everything I've tried returns the this...:

GET(url)

Response [https://www.blackrock.com/ca/individual/en/site-entry?targetUrl=%2Fca%2Findividual%2Fen%2Fproducts%2Fproduct-list%3FsiteEntryPassthrough%3Dtrue]
  Status: 200
  Content-type: text/html;charset=UTF-8
  Size: 270 kB
<!DOCTYPE html><html xmlns="http://www.w3.org/1999/xhtml" prefix="og: http://ogp.me/ns#" lang="en" xml:lang="en"><head><meta content="text/html;charset=UTF-8" http-equiv="Content-type...
<meta name="description" content="" />
<meta name="robots" content="noindex,noarchive,nocache" />
<meta property="og:title" content="Site Entry" />
<meta property="og:type" content="website" />
<meta property="og:image" content="/amer-retail-assets/include/common/images/blackrock_logo.png" />
<meta property="og:site_name" content="BlackRock" />
<meta property="og:locale" content="en_CA" />
<meta property="og:url" content="https://www.blackrock.com/ca/individual/en/site-entry?locale=en_CA" />
<link rel="canonical" href="https://www.blackrock.com/ca/individual/en/site-entry?locale=en_CA" />

So, instead I tried copying the cookie after manually passing the gateway or site-entry page in chrome:

cookie = c(`JSESSION_amer-retail01` = "AC91C17F269D8F6A136E576531FA6E05",
`UnicaID` = "10.39.18.249-1408392737241856",
`UnicaNIODID` = "NQHKNKQYKDm-YxUEYgX",
`_ga` = "GA1.2.127078965.1408392737",
`blkUserType-ca-one` = "institutional",
`s_pers` = "%20s_fid%3D14569E53C0F4EE51-3982590542B2DDA4%7C1471566685393%3B%20gpv%3Dca-one%257Cproducts%257Cproduct%2520list%7C1408410085400%3B%20s_nr%3D1408408285405-Repeat%7C1439944285405%3B%20s_pers_eVar15%3Dprospect%7C1411000285408%3B%20s_pers_prop19%3Danonymous%7C1411000285411%3B",
`s_sess` = "%20s_cc%3Dtrue%3B%20s_sq%3D%3B", 
`s_vi` = "[CS]v1|29F92F108507B6C6-6000010A8000E2ED[CE]",
`ts-ca-one-locale` = "en_CA")

GET(url, set_cookies(.cookies = cookie), user_agent("Mozilla/5.0"))

This returns the same result.

However, when I look at the examples for httr I think the results I'm getting for setting the cookie are incorrect.

For example:

> example(set_cookies)

st_cks> set_cookies(a = 1, b = 2)
Config: 
List of 1
 $ cookie:"a1=;b2="

st_cks> set_cookies(.cookies = c(a = "1", b = "2"))
Config: 
List of 1
 $ cookie:"a1=;b2="

st_cks> GET("http://httpbin.org/cookies")
Response [http://httpbin.org/cookies]
  Status: 200
  Content-type: application/json
  Size: 19 B
{
  "cookies": {}

st_cks> GET("http://httpbin.org/cookies", set_cookies(a = 1, b = 2))
Response [http://httpbin.org/cookies]
  Status: 200
  Content-type: application/json
  Size: 50 B
{
  "cookies": {
    "a1": "", 
    "b2": ""
  }

I would have expected httpbin.org to return a = 1 and b = 1 instead of a1 = and b2 =. Am I doing something wrong?

回答1:

You're not doing anything wrong (at least IMO). Taking a look at set_cookies, there's a bit of code there:

cookie <- paste0(names(cookies), cookies_str, sep = "=", collapse = ";")

which is not working as intended (i.e. it's turning a = "1" into a1= instead of a=1).

You can use this one:

sc2 <- function (..., .cookies = character(0)) {

  cookies <- c(..., .cookies)
  stopifnot(is.character(cookies))
  cookies_str <- vapply(cookies, RCurl::curlEscape, FUN.VALUE = character(1))
  cookie <- paste(names(cookies), cookies_str, sep = "=", 
                   collapse = ";")
  config(cookie = cookie)

}

GET("http://httpbin.org/cookies", sc2(a = 1, b = 2))

## Response [http://httpbin.org/cookies]
##   Status: 200
##   Content-type: application/json
##   Size: 50 B
## {
##   "cookies": {
##     "a": "1", 
##     "b": "2"
##   }

that uses paste instead of paste0 temporarily until that's fixed (I'll file an issue on the httr github repo).

来源：https://stackoverflow.com/questions/25374231/copying-cookie-for-httr

标签

httr