问题
I'm checking a python library: requests-html. Looks interesting, easy and clear scraping. However, I'm not sure how to render a page with infinite scrolling.
From their documentation I understand that I should render a page with special attribute (scrolldown). I'm trying but I do not know how exactly. I know how to use selenium to handle infinite scroll, but I wonder whether it is possible with requests-html.
from requests_html import HTML, HTMLSession
page1 = session.get(url1)
page1.html.render( scrolldown=5,sleep=3)
html = HTML(html=page1.text)
noticeName = html.find('h2.noticeName')
for element in noticeName:
print(element.text)
It finds 10 elements from 13. 10 is visible without scrolling (and loading new content because of infinite scroll).
回答1:
scrolldown=5 means scroll 5 pixel down, is your monitor that small?? or vm height that small?? now give it a bigger value like height of the screen with sleep or 2000 or 5000 without sleep
And it will not give you uniquely next elements, it will give you exactly all elements from the starting.
I will add some sample code soon.
来源：https://stackoverflow.com/questions/56586016/requests-html-and-infinite-scrolling