is it possible to write web crawler in javascript?

后端 未结 11 586
深忆病人
深忆病人 2021-02-01 07:48

I want to crawl the page and check for the hyperlinks in that respective page and also follow those hyperlinks and capture data from the page

11条回答
  •  醉酒成梦
    2021-02-01 08:45

    My typical setup is to use a browser extension with cross origin privileges set, which is injecting both the crawler code and jQuery.

    Another take on Javascript crawlers is to use a headless browser like phantomJS or casperJS (which boosts phantom's powers)

提交回复
热议问题