Scraping Instagram followers page using selenium and python

Question

I have a question related to scraping the instagram followers page. I have a code but it displays only 9 followers. Kindly help me.

from selenium import webdrive         


        

Answer 1

You can easily scroll down using javascript by increasing the scrollTop. You run this scroll until the amount of users in the list no longer changes.

The difference in the amount of users can be checked using the following function

count = 0

def check_difference_in_count(driver):
    global count

    new_count = len(driver.find_elements_by_xpath("//div[@role='dialog']//li"))

    if count != new_count:
        count = new_count
        return True
    else:
        return False

And the following script scrolls down the user container until it has reached the bottom

while 1:
    # scroll down
    driver.execute_script("document.querySelector('div[role=dialog] ul').parentNode.scrollTop=1e100")

    try:
        WebDriverWait(driver, 5).until(check_difference_in_count)
    except:
        break