How to implement cookie support in ruby net/http?

Question

I\'d like to add cookie support to a ruby class utilizing net/http to browse the web. Cookies have to be stored in a file to survive after the script has ended. Of course I

Answer 1

I've used Curb and Mechanize for a similar project. Just enable cookies support and save the cookies to a temp cookiejar... If your using net/http or packages without cookie support built in, you will need to write your own cookie handling.

Answer 2

Taken from DZone Snippets

http = Net::HTTP.new('profil.wp.pl', 443)
http.use_ssl = true
path = '/login.html'

# GET request -> so the host can set his cookies
resp, data = http.get(path, nil)
cookie = resp.response['set-cookie'].split('; ')[0]


# POST request -> logging in
data = 'serwis=wp.pl&url=profil.html&tryLogin=1&countTest=1&logowaniessl=1&login_username=blah&login_password=blah'
headers = {
  'Cookie' => cookie,
  'Referer' => 'http://profil.wp.pl/login.html',
  'Content-Type' => 'application/x-www-form-urlencoded'
}

resp, data = http.post(path, data, headers)


# Output on the screen -> we should get either a 302 redirect (after a successful login) or an error page
puts 'Code = ' + resp.code
puts 'Message = ' + resp.message
resp.each {|key, val| puts key + ' = ' + val}
puts data

update

#To save the cookies, you can use PStore
cookies = PStore.new("cookies.pstore")

# Save the cookie  
cookies.transaction do
  cookies[:some_identifier] = cookie
end

# Retrieve the cookie back
cookies.transaction do
  cookie = cookies[:some_identifier] 
end

Answer 3

You can send receive cookies using headers.

You can store the header in any persistence framework. Whether it is some sort of database, or files.

Answer 4

Use http-cookie, which implements RFC-compliant parsing and rendering, plus a jar.

A crude example that happens to follow a redirect post-login:

require 'uri'
require 'net/http'
require 'http-cookie'

uri = URI('...')
jar = HTTP::CookieJar.new

Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') do |http|
  req = Net::HTTP::Post.new uri
  req.form_data = { ... }
  res = http.request req
  res.get_fields('Set-Cookie').each do |value|
    jar.parse(value, req.uri)
  end

  fail unless res.code == '302'

  req = Net::HTTP::Get.new(uri + res['Location'])
  req['Cookie'] = HTTP::Cookie.cookie_value(jar.cookies(uri))
  res = http.request req
end

Why do this? Because the answers above are incredibly insufficient and flat out don't work in many RFC-compliant scenarios (happened to me), so relying on the very lib implementing just what's needed is infinitely more robust if you want to handle more than one particular case.

Answer 5

The accepted answer does not work. You need to access the internal representation of the response header where the multiple set-cookie values are stores separately and then remove everything after the first semicolon from these string and join them together. Here is code that works

r = http.get(path)
cookie = {'Cookie'=>r.to_hash['set-cookie'].collect{|ea|ea[/^.*?;/]}.join}
r = http.get(next_path,cookie)

Answer 6

The accepted answer will not work if your server returns and expects multiple cookies. This could happen, for example, if the server returns a set of FedAuth[n] cookies. If this affects you, you might want to look into using something along the lines of the following instead:

http = Net::HTTP.new('https://example.com', 443)
http.use_ssl = true
path1 = '/index.html'
path2 = '/index2.html'

# make a request to get the server's cookies
response = http.get(path)
if (response.code == '200')
    all_cookies = response.get_fields('set-cookie')
    cookies_array = Array.new
    all_cookies.each { | cookie |
        cookies_array.push(cookie.split('; ')[0])
    }
    cookies = cookies_array.join('; ')

    # now make a request using the cookies
    response = http.get(path2, { 'Cookie' => cookies })
end