Scrapy: Define items dynamically

Question

As I started to learn scrapy, i have come accross a requirement to dynamically build the Item attributes. I\'m just scraping a webpage which has a table structure and I wanted t

Answer 1

I was more xpecting about explanation in handling the data with item loaders and pipelines

Assuming:

fieldname = 'test'
fieldxpath = '//h1'

It's (in recent versions) very simple...

item = Item()
l = ItemLoader(item=item, response=response)

item.fields[fieldname] = Field()
l.add_xpath(fieldname, fieldxpath)

return l.load_item()