Scrape dynamic data using scrapy [closed]

问题

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 1 year ago.

Improve this question

I would like to scrape option chain of stock from nasdaq website using scrapy (along with other data)

Nasdaq recently updated their website. Here is the url I am talking about.

The data is not loaded with plain spider and in scrapy shell. From the scrapy docs, I need to use scrapy-splash or scrapy-selenium.

I would like to know which one is the right tool for the job before investing my time. Or is there something else that is recommended?

Thanks!

回答1:

For that task you don't need selenium or splash. Instead you can inspect the network tab in chrome developer tools.

The site makes additional request to get data from the following endpoint: https://api.nasdaq.com/api/quote/MSFT/option-chain?assetclass=stocks&limit=0

Cheers

来源：https://stackoverflow.com/questions/57573886/scrape-dynamic-data-using-scrapy