Scrapy spider not found error

Question

This is Windows 7 with python 2.7

I have a scrapy project in a directory called caps (this is where scrapy.cfg is)

My spider is located in caps\\caps\\spiders\\c

Answer 1

Make sure you have set the "name" property of the spider. Example:

class campSpider(BaseSpider):
   name = 'campSpider'

Without the name property, the scrapy manager will not be able to find your spider.

Answer 2

Also make sure that your project is not called scrapy! I made that mistake and renaming it fixed the problem.

Answer 3

Ahh Yes, you should enter the value of your 'name variable value'.

I.e.

import scrapy

class QuoteSpider(scrapy.Spider):
    name = 'quotes'
    start_urls = [
        'http://quotes.toscrape.com/'
    ]

    def parse(self, response):
        title = response.css('title').extract()
        yield {'titleText' : title}

So in this case, the name = 'quotes'. Then in your command line you enter: 'scrapy crawl quotes'

That was my problem.

Answer 4

Check indentation too, the class for my spider was indented one tab. Somehow that makes the class invalid or something.

Answer 5

I had the same issue. When i was using "scrapy list" in cmd the command listed the spider name i was getting the error for, in the list, but while i tried to run it with scrapy crawl SpiderName.py, i used to get Scrapy spider not found error. I have used this spider before and everything was fine with it. So i used the secret weapon, i restarted my system and the issue was resolved :D

Answer 6

Ensure same name attribute is used in the command line for running spider ...

scrapy crawl