I got too many 429 errors when I enabled cache extension and proxy middleware in the same time in scrapy

前端 未结 0 1592
谎友^
谎友^ 2021-02-01 17:28

I am using scrapy to crawl data, the target website will block IP when it sent about 1000 requests, so I write a proxy middleware, and because the amount of data is relatively l

相关标签:
回答
  • 消灭零回复
提交回复
热议问题