How do I use cookies with RCurl?

Question

I am trying to write an R package that accesses some data via a REST API. The API, however, doesn\'t use http authentication, but rather relies on cookies to keep credential

Answer 1

My bad. Neal Richter pointed out to me http://www.omegahat.org/RCurl/RCurlJSS.pdf - which better explains the difference between cookiefile and cookiejar. The sample script in the question actually does work. But it only writes the file to disk when it is no longer being used.

Answer 2

In general you don't need to create a cookie file, unless you want to study the cookies.

Given this, in real word, web servers use agent data, redirecting and hidden post data, but this should help:

library(RCurl)

#Set your browsing links 
loginurl = "http://api.my.url/login"
dataurl  = "http://api.my.url/data"

#Set user account data and agent
pars=list(
     username="xxx"
     password="yyy"
)
agent="Mozilla/5.0" #or whatever 

#Set RCurl pars
curl = getCurlHandle()
curlSetOpt(cookiejar="cookies.txt",  useragent = agent, followlocation = TRUE, curl=curl)
#Also if you do not need to read the cookies. 
#curlSetOpt(  cookiejar="", useragent = agent, followlocation = TRUE, curl=curl)

#Post login form
html=postForm(loginurl, .params = pars, curl=curl)

#Go wherever you want
html=getURL(dataurl, curl=curl)

#Start parsing your page
matchref=gregexpr("... my regexp ...", html)

#... .... ...

#Clean up. This will also print the cookie file
rm(curl)
gc()

Important

There can often be hidden post data, beyond username and password. To capture it you may want, e.g. in Chrome, to use Developer tools (Ctrl Shift I) -> Network Tab, in order to show the post field names and values.