Close a scrapy spider when a condition is met and return the output object

后端 未结 1 1189
慢半拍i
慢半拍i 2021-01-22 11:51

I have made a spider to get reviews from a page like this here using scrapy. I want product reviews only till a certain date(2nd July 2016 in this case). I want to close my spid

相关标签:
1条回答
  • 2021-01-22 12:02

    To force spider to close you can use raise CloseSpider exception as described here in scrapy docs. Just be sure to return/yield your items before you raise the exception.

    0 讨论(0)
提交回复
热议问题