Scraping data without having to explicitly define each field to be scraped

Question

I want to scrape a page of data (using the Python Scrapy library) without having to define each individual field on the page. Instead I want to dynamically generate fields using

Answer 1

This solution works with the exporters (scrapy crawl -t json -o output.json):

import scrapy

class FlexibleItem(scrapy.Item):
    def __setitem__(self, key, value):
        if key not in self.fields:
            self.fields[key] = scrapy.Field()
        super(FlexibleItem, self).__setitem__(key, value)

EDIT: updated to work with latest Scrapy