Pagination Loop Google Script

问题

I'm a total novice and have come up with the following Google Script to send a GET Request and then parse the response into a Google Sheet.

I am only able to request 50 items on the first page and I have tried to search for a way to loop pages until I get all of the data that I requested.

I'm sure this is pretty easy if somebody could point me in the right direction? :-)

function myData1() {
  // Call the horizon API
  var url = "https://apc.hypaship.com/api/3.0/Orders.json?datefrom=27-03-2018T00:01";

  var headers = {"Content-Type": "application/json",
                 "remote-user": "Basic ****************************"};

  var options = {"method" : "get",
                 "headers" : headers};

  var response = UrlFetchApp.fetch(url,options);
  var text = response.getResponseCode();

  Logger.log(response);

  // Parse the JSON reply
  var json = response.getContentText();
  var data = JSON.parse(json);

  //Loop for data required
  for (i = 0; i <= data.Orders.Order.length - 1; i++) {
  var row = data.Orders.Order[i];
  var customer = row.Collection.CompanyName;
  var pcde = row.Collection.PostalCode;

  Logger.log(customer);
  Logger.log(pcde);

  //Export to Google Sheet
  var sheet = SpreadsheetApp.getActiveSheet();
  sheet.getRange(sheet.getLastRow() + 1,1).setValue([customer]);
  sheet.getRange(sheet.getLastRow() + 0,2).setValue([pcde]);}
}

{"Orders": {"AccountNumber": "279", "Messages": {"Code": "SUCCESS", "Description": "SUCCESS"}, "Pagination": {"TotalPages": "1", "NextPage": null, "PreviousPage": null, "ItemsPerPage": "50", "ItemsTotal": "4"},

[18-03-27 16:51:46:895 BST] {"Orders": {"AccountNumber": "279", "Messages": {"Code": "SUCCESS", "Description": "SUCCESS"}, "Pagination": {"TotalPages": "23", "NextPage": "2", "PreviousPage": null, "ItemsPerPage": "50", "ItemsTotal": "1121"},

回答1:

Since the API you are working with provides both the total number of pages, and the next page, you can use any loop structure you are comfortable with. Note, depending on time needed for these queries and whatever is done with their results, you may need to adopt a resumable approach.

In general, I recommend either the do while or explicit for loop approaches:

function doWhileExample() {
  var url = /* my api url */;
  var headers = ...;
  var options = ...;
  // An array of query parameters that are static for this particular request.
  var queryParams = [
    "datefrom=" + /* value of this param */,
    /* other params */
  ];
  var pageKey = /* the string keyword for specifying the result page for your API. Could be "page". */;
  var pageNum; /* could read a stored page number, e.g. from cache / properties service */

  // Execute your query / queries.
  var data = {}, output = [];
  do {
    var currentUrl = url + "?" + queryParams.join("&");
    if (pageNum) {
      currentUrl += "&" + pageKey + "=" + pageNum;
    }
    var response = UrlFetchApp.fetch(currentUrl, options);
    var code = response.getResponseCode();
    /* do some error checking on the code e.g. don't try to parse data if the server is dead */

    // Store data for batch serialization after requests are done.
    // (Assumes you store "rectangular" data.)
    data = JSON.parse(response.getContentText());
    for (var r = 0, rows = data.Orders.Order.length; r < rows; ++r) {
      var row = data.Orders.Order[r];
      output.push([row.Some.Data.One, row.Some.Data.Two, row.Some.Other.Data, ... ]);
    }
    // Update the page number.
    pageNum = data.Orders.Pagination.NextPage;
  } while (pageNum);

  // Serialize all data.
  var sheet = SpreadsheetApp.openById("someId").getSheetByName("someName");
  if (sheet && output.length && output[0].length) {
    sheet.getRange(sheet.getLastRow(), 1).offset(1, 0, output.length, output[0].length).setValues(output);
  }
}

As hinted in the code above, you'll need to somehow indicate to the API url that you want a specific page, e.g. &page=3 would work if it accepts those parameters in the url (and page is the right keyword) - it might want this kind of request manipulation in a header, etc.

来源：https://stackoverflow.com/questions/49514904/pagination-loop-google-script

标签

google-apps-script