Scraping Instagram followers page using selenium and python

Question

I have a question related to scraping the instagram followers page. I have a code but it displays only 9 followers. Kindly help me.

from selenium import webdrive         


        

Answer 1

You have to add one for loop so that you can scroll down the page for the followers. This for loop can be like this:

#Find the followers page
dialog = driver.find_element_by_xpath('/html/body/div[2]/div/div[2]/div/div[2]')
#find number of followers
allfoll=int(driver.find_element_by_xpath("//li[2]/a/span").text) 
#scroll down the page
for i in range(int(allfoll/2)):
    driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", dialog)
    time.sleep(random.randint(500,1000)/1000)
    print("Extract friends %",round((i/(allfoll/2)*100),2),"from","%100")

Answer 2

You can easily scroll down using javascript by increasing the scrollTop. You run this scroll until the amount of users in the list no longer changes.

The difference in the amount of users can be checked using the following function

count = 0

def check_difference_in_count(driver):
    global count

    new_count = len(driver.find_elements_by_xpath("//div[@role='dialog']//li"))

    if count != new_count:
        count = new_count
        return True
    else:
        return False

And the following script scrolls down the user container until it has reached the bottom

while 1:
    # scroll down
    driver.execute_script("document.querySelector('div[role=dialog] ul').parentNode.scrollTop=1e100")

    try:
        WebDriverWait(driver, 5).until(check_difference_in_count)
    except:
        break