Feedparser to dataframe doesnt ouput all columns

Question

I parsing a URL from feedparser and trying to get all columns, but i do not get all columns as a out put, not sure where the issue is. If you execute the below. I do not get dat

Answer 1

Another method.

import pandas as pd
from simplified_scrapy import SimplifiedDoc, utils, req

getdeals = ['http://www.ebay.com/rps/feed/v1.1/epnexcluded/EBAY-US?limit=200',
            'http://www.ebay.com/rps/feed/v1.1/epnexcluded/EBAY-US?limit=200&offset=200',
            'http://www.ebay.com/rps/feed/v1.1/epnexcluded/EBAY-US?limit=200&offset=400']
    
posts=[]
header = ['title','endsAt','image255','price','originalPrice','discountPercentage','shippingCost','dealUrl']
for url in getdeals:
    try: # It's a good habit to have try and exception in your code.
        feed = SimplifiedDoc(req.get(url))
        for deals in feed.selects('item'):
            row = []
            for h in header: row.append(deals.select(h+">text()")) # Returns None when the element does not exist
            posts.append(row)
    except Exception as e:
        print (e)
        
df=pd.DataFrame(posts,columns=header)
df.tail()