infinite-scroll

How to make the Apify Crawler to scroll full page when web page have infinite scrolling?

a 夏天 提交于 2021-02-19 16:12:03
问题 I'm facing a problem that I unable to get all the product data as the website using a lazy load on product catalog page. meaning it needs to scroll until the whole page loaded. I getting only first-page products data. 回答1: First, you should keep in mind that there are infinite ways that infinite scroll can be implemented. Sometimes you have to click buttons on the way or do any sort of transitions. I will cover only the most simple use-case here which is scrolling down with some interval and

How to make the Apify Crawler to scroll full page when web page have infinite scrolling?

∥☆過路亽.° 提交于 2021-02-19 16:08:45
问题 I'm facing a problem that I unable to get all the product data as the website using a lazy load on product catalog page. meaning it needs to scroll until the whole page loaded. I getting only first-page products data. 回答1: First, you should keep in mind that there are infinite ways that infinite scroll can be implemented. Sometimes you have to click buttons on the way or do any sort of transitions. I will cover only the most simple use-case here which is scrolling down with some interval and

How to make the Apify Crawler to scroll full page when web page have infinite scrolling?

妖精的绣舞 提交于 2021-02-19 16:05:14
问题 I'm facing a problem that I unable to get all the product data as the website using a lazy load on product catalog page. meaning it needs to scroll until the whole page loaded. I getting only first-page products data. 回答1: First, you should keep in mind that there are infinite ways that infinite scroll can be implemented. Sometimes you have to click buttons on the way or do any sort of transitions. I will cover only the most simple use-case here which is scrolling down with some interval and

How to make the Apify Crawler to scroll full page when web page have infinite scrolling?

生来就可爱ヽ(ⅴ<●) 提交于 2021-02-19 16:04:43
问题 I'm facing a problem that I unable to get all the product data as the website using a lazy load on product catalog page. meaning it needs to scroll until the whole page loaded. I getting only first-page products data. 回答1: First, you should keep in mind that there are infinite ways that infinite scroll can be implemented. Sometimes you have to click buttons on the way or do any sort of transitions. I will cover only the most simple use-case here which is scrolling down with some interval and

Retrieve HTML content from an infinite scroll page (Facebook)

对着背影说爱祢 提交于 2021-02-10 17:31:49
问题 I would like tor retrieve HTML data from a dynamic web page, like for example a public Facebook page: https://www.facebook.com/bbcnews/ (public content, without login) For example, in this page, we have an infinite scroll, and we have to go at the bottom of the page to load more posts. My current code is here: URL url = new URL("https://www.facebook.com/bbcnews/"); BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream())); BufferedWriter writer = new BufferedWriter

Infinite vertical scrollview both ways (add items dynamically at top/bottom) that doesn’t interfere with scroll position when you add to list start

岁酱吖の 提交于 2021-01-28 19:37:02
问题 I’m after a vertical scrollview that’s infinite both ways: scrolling up to the top or down to the bottom results in more items being added dynamically. Almost all help I’ve encountered is only concerned with the bottom side being infinite in scope. I did come across this relevant answer but it’s not what I’m specifically looking for (it’s adding items automatically based on time duration, and requires interaction with direction buttons to specify which way to scroll). This less relevant

CSS3 transform: translate maximum value?

陌路散爱 提交于 2021-01-27 13:17:46
问题 I created an experiment to infinite-scroll the first billion digits of Pi to find/create a scrolling solution that has high-performance with a massive dataset. I started testing with iScroll and ran into an issue. This demo works great (in Chrome) till around 33 million transform: translate(0px, 3.35545e+07px); You can see the issue by running the following commands in the dev tools console, then scrolling. scroller.scrollTo(0, -33553700); scroller._execEvent('scroll'); Any CSS experts know

ionic 3 infinite scroll is not working until content scroll to top and then bottom

拜拜、爱过 提交于 2021-01-02 20:28:17
问题 I am trying to load some more data in ionic 3 project. The infinite scroll is working only for the first time. And then it is stop working until the page scroll to top and then bottom. Here is my code HTML <ion-content> <ion-scroll style="width:100%;height:100vh" scrollY="true"> <ion-list *ngIf="posts.length>0"> <button *ngFor="let post of posts" ion-item> <ion-thumbnail item-start> <img [src]="getPostImage(post)"> </ion-thumbnail> <h2>{{ post.title.rendered }}</h2> </button> </ion-list> <ion

Angular virtual scroll: append new items when reaching the end of scroll

馋奶兔 提交于 2020-12-29 09:17:04
问题 I would like to use virtual scroll on my Angular application. The items in my list are the result of a remote paged search. I would like to load more results (call the next page) every time I reach the end of the viewport scrolling down. Here is my template: <div class="container"> <cdk-virtual-scroll-viewport itemSize="100"> <li *cdkVirtualFor="let item of searchResult">{{item}}</li> </cdk-virtual-scroll-viewport> </div> Here is my attempts to make it to work as I need: export class

Driver doesn't return proper page source

本小妞迷上赌 提交于 2020-12-08 02:35:54
问题 I'm trying to load one web page. Then scroll to the very bottom of this page (there is an infinite scroll) and get a page source code. Scrolling and loading seems to work correct but driver.page_source returns very short html which is just a little part of the whole page source . def scroll_to_the_bottom(driver): old_html = '' new_html = driver.page_source while old_html != new_html: print 'SCROLL' old_html = driver.page_source driver.execute_script("window.scrollTo(0, document.body