Scraping data without having to explicitly define each field to be scraped

Question

I want to scrape a page of data (using the Python Scrapy library) without having to define each individual field on the page. Instead I want to dynamically generate fields using

Answer 1

This works with version 0.24 and also allows Items to work with Item Loaders:

import scrapy
from collections import defaultdict

class FlexibleItem(scrapy.Item):
    fields = defaultdict(scrapy.Field)

    def __setitem__(self, key, value):
        # all keys are supported
        self._values[key] = value