How to read a file downloaded by selenium webdriver in python

Question

I am using selenium with webdriver in python to download a csv file from a site . The file gets downloaded into the download directory specified. Here is an overview of my c

Answer 1

This answer was formed from a combination of previous stack overflow questions , answers as well as comments in this post so thank you everyone.

I combined selenium webdriver and the python requests module for this solution . I essentially logged into the site using selenium, copied the cookies from the webdriver session and then used a requests.get(url,cookies = webdriver_cookies) to get the file.

Here's the gist of my solution

fp = webdriver.FirefoxProfile() 
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False) 
fp.set_preference("browser.download.dir",'xx/yy') 
fp.set_preference('browser.helperApps.neverAsk.saveToDisk', "text/plain, application/vnd.ms-excel, text/csv, text/comma-separated-values, application/octet-stream") 
driver = webdriver.Firefox(fp)

# selenium login code ...

driver_cookies = driver.get_cookies()
cookies_copy = {}
for driver_cookie in driver_cookies:
    cookies_copy[driver_cookie["name"]] = driver_cookie["value"]
r = requests.get('url',cookies = cookies_copy)
print r.text

I hope that this helps someone