Crawling LinkedIn while authenticated with Scrapy

Question

So I\'ve read through the Crawling with an authenticated session in Scrapy and I am getting hung up, I am 99% sure that my parse code is correct, I just don\'t believe the l

Answer 1

class LinkedPySpider(BaseSpider):

should be:

class LinkedPySpider(InitSpider):

Also you shouldn't override the parse function as I mentioned in my answer here: https://stackoverflow.com/a/5857202/crawling-with-an-authenticated-session-in-scrapy

If you don't understand how to define the rules for extracting links, just have a proper read through the documentation:
http://readthedocs.org/docs/scrapy/en/latest/topics/spiders.html#scrapy.contrib.spiders.Rule
http://readthedocs.org/docs/scrapy/en/latest/topics/link-extractors.html#topics-link-extractors