How to get data from all pages in Github API with Python?

Question

I\'m trying to export a repo list and it always returns me information about the 1rst page. I could extend the number of items per page using URL+\"?per_page=100\" but it\'s not

Answer 1

First you use

print(a.headers.get('link'))

this will give you the number of pages the repository has, similar to below

; rel="next", 

; rel="last"

from this you can see that currently we are on first page of repo, rel='next' says that the next page is 2, and rel='last' tells us that your last page is 8.

After knowing the number of pages to traverse through,you just need to use '=' for page number while getting request and change the while loop until the last page number, not len(repo) as it will return you 100 each time. for e.g

i=1
while i <= 8:
      r = requests.get('https://api.github.com/orgs/xxxx/repos?page={0}&type=all'.format(i),
                         auth=('My_user', 'My_passwd'))
      repo = r.json()
      for j in repo:
        print(repo[j][u'full_name'])
      i = i + 1