Using phantomjs for dynamic content with scrapy and selenium possible race condition

Question

First off, this is a follow up question from here: Change number of running spiders scrapyd

I\'m used phantomjs and selenium to create a downloader middleware for my scr

Answer 1

try using webdriver to interface with phantomjs https://github.com/brandicted/scrapy-webdriver