Scrapy request+response+download time

Question

UPD: Not close question because I think my way is not so clear as should be

Is it possible to get current request + response + download time for saving

Answer 1

Not sure if you need a Middleware here. Scrapy has a request.meta which you can query and yield. For download latency, simply yield

download_latency=response.meta.get('download_latency'),

The amount of time spent to fetch the response, since the request has been started, i.e. HTTP message sent over the network. This meta key only becomes available when the response has been downloaded. While most other meta keys are used to control Scrapy behavior, this one is supposed to be read-only.