How to make mechanize wait for web-page 'full' load?

前端 未结 2 827
半阙折子戏
半阙折子戏 2021-02-20 07:47

I want to scrape some web page which loads its components dynamically. This page has an onload script, and I can see the complete page 3-5 seconds after typing the URL into my b

2条回答
  •  悲&欢浪女
    2021-02-20 07:52

    The problem you're having is that the web page is rendered in your web browser through the javascript engine. However, mechanize doesn't have the ability to execute javascript on its own so, no matter how long you wait, you aren't going to get the HTML you're missing using just mechanize.

    For more information about how scrape dynamically generated content, please have a look at this question.

提交回复
热议问题