I am attempting to set up a custom downloader middleware class in Scrapy. I suspect that I\'ve missed something obvious, but I\'ve read over the docs a few times and have found
DOWNLOADER_MIDDLEWARES = { 'myproject.middlewares.TestDownloader': 400 }
For this to work, create file middlewares.py
inside myproject
folder, and in that file put your downloader middleware class called TestDownloader
.
Or having middlewares
folder with __init__.py
inside it, you can put put your downloader middleware class called TestDownloader
inside __init__.py
-- this should work too.