I am trying to grab div\'s with the class: \'product\'. The problem is, some of the div\'s with class \'product\' also have the class \'product-small\'. So when I use xpat
You should consider using a CSS selector for this part of your query.
http://doc.scrapy.org/en/latest/topics/selectors.html#when-querying-by-class-consider-using-css
from scrapy import Selector
sel = Selector(text='I am a product!')
print sel.css('.product').extract()
If you need to, you can chain CSS and XPath selectors, as in the example on that page.