NextPageToken gives invalid input error, 400 - Google Directory API ChomeDevices

问题

I have to fetch 250K chromebooks from google workspace (Gsuite), I am using Admin Directory API to retrieve JSON data from Google.

The response returns in chunks of 200 records, in the response is included a nextPageToken, I use that next page token to retrieve the next 200 and so on.

After an hour, of using the nextPageToken attached from the previous request, However Google returns with error 400,

{error_code: 400, "message"=>"Invalid Input: CMiJhq7-5ewCEp0BCm737N8GN......"},

Note: This string 'CMiJhq7-5ewCEp0BCm737N8GN......' which google is calling as invalid is the nextPageToken.

Why is this happening? Does nextPageToken expire after 1 hour?

My code snippet:

query_list = {
      'maxResults' => 200,
      'access_token' => access_token,
      'pageToken' => next_page_token
    }
HTTParty.get(endpoint_url, query: query_list)

回答1:

The nextPage token is created when the initial request is sent. This token is used in order to get the next batch of rows from the request.

This token is intended to be used immediately as the data associated with the initial request may be changed if you wait to long.

So yes next page tokens do expire i would actually expect them to expire in a lot less than an hour. I also wonder if the next page token wouldn't just expire after you used it the first time.

If you want to make the same request again i suggest you do that and get new next page tokens built for you after the hour.

回答2:

I had to change my approach, initially, I fetched 200 chunks from Google API, performed some time-taking processing and then made entries into my database (database-intensive tasks) and then requested the next 200 chunks and so so. After an hour, the last nextpagetoken sent by Google became invalid.

So, now I fetch 200 chunks, save them to my database in JSON format without performing any database-intensive tasks, request the next 200, and so on. I was able to fetch 300K Chromebooks JSON data from google in around 56 Minutes before the nextPageToken became invalid.

I am now processing that JSON data present in my database, without having network overhead or any google API dependency.

来源：https://stackoverflow.com/questions/64661575/nextpagetoken-gives-invalid-input-error-400-google-directory-api-chomedevices