Scrapy CrawlSpider + Splash: how to follow links through linkextractor?

Question

I have the following code that is partially working,

class ThreadSpider(CrawlSpider):
    name = \'thread\'
    allowed_domains = [\'bbs.example.com\']
    star         


        

Answer 1

Use below code - Just copy and paste

restrict_xpaths=('//a[contains(text(), "Next Page")]')

Instead of

restrict_xpaths=("//a[contains(text(), 'Next Page')]")