Can scrapy be used to scrape dynamic content from websites that are using AJAX?

前端 未结 8 783
星月不相逢
星月不相逢 2020-11-21 17:48

I have recently been learning Python and am dipping my hand into building a web-scraper. It\'s nothing fancy at all; its only purpose is to get the data off of a betting we

8条回答
  •  一向
    一向 (楼主)
    2020-11-21 18:25

    I handle the ajax request by using Selenium and the Firefox web driver. It is not that fast if you need the crawler as a daemon, but much better than any manual solution. I wrote a short tutorial here for reference

提交回复
热议问题