Scrapy: catch responses with specific HTTP server codes

Question

We have a pretty much standard Scrapy project (Scrapy 0.24).

I\'d like to catch specific HTTP response codes, such as 200, 500, 502, 503, 504 etc.

Something like

Answer 1

By default, Scrapy only handles responses with status codes 200-300.

Let Scrapy handle 500 and 502:

class Spider(...):
    handle_httpstatus_list = [500, 502]

Then, in the parse() callback, check response.status:

def parse(response):
    if response.status == 500:
        # logic here
    elif response.status == 502:
        # logic here