Scrapy:In a request fails (eg 404,500), how to ask for another alternative request?

后端 未结 2 1367
借酒劲吻你
借酒劲吻你 2021-02-15 18:18

I have a problem with scrapy. In a request fails (eg 404,500), how to ask for another alternative request? Such as two links can obtain price info, the one failed, request anoth

2条回答
  •  走了就别回头了
    2021-02-15 19:05

    Use "errback" in the Request like errback=self.error_handler where error_handler is a function (just like callback function) in this function check the error code and make the alternative Request.

    see errback in the scrapy documentation: http://doc.scrapy.org/en/latest/topics/request-response.html

提交回复
热议问题