How to use R to download a zipped file from a SSL page that requires cookies

Question

I am trying to download a file from an https page that requires an \"I Agree\" button be pushed and then stores a cookie. My apologies if this answer is obvious somewhere..

Answer 1

This is a bit easier to do with httr because it sets up everything so that cookies and https work seamlessly.

The easiest way to generate the cookies is to have the site do it for you, by manually posting the information that the "I agree" form generates. You then do a second request to download the actual file.

library(httr)
terms <- "http://www.icpsr.umich.edu/cgi-bin/terms"
download <- "http://www.icpsr.umich.edu/cgi-bin/bob/zipcart2"

values <- list(agree = "yes", path = "SAMHDA", study = "32722", ds = "", 
  bundle = "all", dups = "yes")

# Accept the terms on the form, 
# generating the appropriate cookies
POST(terms, body = values)
GET(download, query = values)

# Actually download the file (this will take a while)
resp <- GET(download, query = values)

# write the content of the download to a binary file
writeBin(content(resp, "raw"), "c:/temp/thefile.zip")