What’s the best RESTful method to return total number of items in an object?

Question

I’m developing a REST API service for a large social networking website I’m involved in. So far, it’s working great. I can issue GET, POST, P

Answer 1

As of "X-"-Prefix was deprecated. (see: https://tools.ietf.org/html/rfc6648)

We found the "Accept-Ranges" as being the best bet to map the pagination ranging: https://tools.ietf.org/html/rfc7233#section-2.3 As the "Range Units" may either be "bytes" or "token". Both do not represent a custom data type. (see: https://tools.ietf.org/html/rfc7233#section-4.2) Still, it is stated that

HTTP/1.1 implementations MAY ignore ranges specified using other units.

Which indicates: using custom Range Units is not against the protocol, but it MAY be ignored.

This way, we would have to set the Accept-Ranges to "members" or whatever ranged unit type, we'd expect. And in addition, also set the Content-Range to the current range. (see: https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.12)

Either way, I would stick to the recommendation of RFC7233 (https://tools.ietf.org/html/rfc7233#page-8) to send a 206 instead of 200:

If all of the preconditions are true, the server supports the Range
header field for the target resource, and the specified range(s) are
valid and satisfiable (as defined in Section 2.1), the server SHOULD
send a 206 (Partial Content) response with a payload containing one
or more partial representations that correspond to the satisfiable
ranges requested, as defined in Section 4.

So, as a result, we would have the following HTTP header fields:

For Partial Content:

206 Partial Content
Accept-Ranges: members
Content-Range: members 0-20/100

For full Content:

200 OK
Accept-Ranges: members
Content-Range: members 0-20/20

Answer 2

_{I have been doing some extensive research into this and other REST paging related questions lately and thought it constructive to add some of my findings here. I'm expanding the question a bit to include thoughts on paging as well as the count as they are intimitely related.}

Headers

The paging metadata is included in the response in the form of response headers. The big benefit of this approach is that the response payload itself is just the actual data requestor was asking for. Making processing the response easier for clients that are not interested in the paging information.

There are a bunch of (standard and custom) headers used in the wild to return paging related information, including the total count.

X-Total-Count

X-Total-Count: 234

This is used in some APIs I found in the wild. There are also NPM packages for adding support for this header to e.g. Loopback. Some articles recommend setting this header as well.

It is often used in combination with the Link header, which is a pretty good solution for paging, but lacks the total count information.

Link

Link: </TheBook/chapter2>;
      rel="previous"; title*=UTF-8'de'letztes%20Kapitel,
      </TheBook/chapter4>;
      rel="next"; title*=UTF-8'de'n%c3%a4chstes%20Kapitel

I feel, from reading a lot on this subject, that the general consensus is to use the Link header to provide paging links to clients using rel=next, rel=previous etc. The problem with this is that it lacks the information of how many total records there are, which is why many APIs combine this with the X-Total-Count header.

Alternatively, some APIs and e.g. the JsonApi standard, use the Link format, but add the information in a response envelope instead of to a header. This simplifies access to the metadata (and creates a place to add the total count information) at the expense of increasing complexity of accessing the actual data itself (by adding an envelope).

Content-Range

Content-Range: items 0-49/234

Promoted by a blog article named Range header, I choose you (for pagination)!. The author makes a strong case for using the Range and Content-Range headers for pagination. When we carefully read the RFC on these headers, we find that extending their meaning beyond ranges of bytes was actually anticipated by the RFC and is explicitly permitted. When used in the context of items instead of bytes, the Range header actually gives us a way to both request a certain range of items and indicate what range of the total result the response items relate to. This header also gives a great way to show the total count. And it is a true standard that mostly maps one-to-one to paging. It is also used in the wild.

Envelope

Many APIs, including the one from our favorite Q&A website use an envelope, a wrapper around the data that is used to add meta information about the data. Also, OData and JsonApi standards both use a response envelope.

The big downside to this (imho) is that processing the response data becomes more complex as the actual data has to be found somewhere in the envelope. Also there are many different formats for that envelope and you have to use the right one. It is telling that the response envelopes from OData and JsonApi are wildly different, with OData mixing in metadata at multiple points in the response.

Separate endpoint

I think this has been covered enough in the other answers. I did not investigate this much because I agree with the comments that this is confusing as you now have multiple types of endpoints. I think it's nicest if every endpoint represents a (collection of) resource(s).

Further thoughts

We don't only have to communicate the paging meta information related to the response, but also allow the client to request specific pages/ranges. It is interesting to also look at this aspect to end up with a coherent solution. Here too we can use headers (the Range header seems very suitable), or other mechanisms such as query parameters. Some people advocate treating pages of results as separate resources, which may make sense in some use cases (e.g. /books/231/pages/52. I ended up selecting a wild range of frequently used request parameters such as pagesize, page[size] and limit etc in addition to supporting the Range header (and as request parameter as well).

Answer 3

Sometimes frameworks (like $resource/AngularJS) require an array as a query result, and you can't really have a response like {count:10,items:[...]} in this case I store "count" in responseHeaders.

P. S. Actually you can do that with $resource/AngularJS, but it needs some tweaks.

Answer 4

When requesting paginated data, you know (by explicit page size parameter value or default page size value) the page size, so you know if you got all data in response or not. When there is less data in response than is a page size, then you got whole data. When a full page is returned, you have to ask again for another page.

I prefer have separate endpoint for count (or same endpoint with parameter countOnly). Because you could prepare end user for long/time consuming process by showing properly initiated progressbar.

If you want to return datasize in each response, there should be pageSize, offset mentionded as well. To be honest the best way is to repeat a request filters too. But the response became very complex. So, I prefer dedicated endpoint to return count.

<data>
  <originalRequest>
    <filter/>
    <filter/>
  </originalReqeust>
  <totalRecordCount/>
  <pageSize/>
  <offset/>
  <list>
     <item/>
     <item/>
  </list>
</data>

Couleage of mine, prefer a countOnly parameter to existing endpoint. So, when specified the response contains metadata only.

endpoint?filter=value

<data>
  <count/>
  <list>
    <item/>
    ...
  </list>
</data>

endpoint?filter=value&countOnly=true

<data>
  <count/>
  <!-- empty list -->
  <list/>
</data>

Answer 5

I would recommend adding headers for the same, like:

HTTP/1.1 200

Pagination-Count: 100
Pagination-Page: 5
Pagination-Limit: 20
Content-Type: application/json

[
  {
    "id": 10,
    "name": "shirt",
    "color": "red",
    "price": "$23"
  },
  {
    "id": 11,
    "name": "shirt",
    "color": "blue",
    "price": "$25"
  }
]

For details refer to:

https://github.com/adnan-kamili/rest-api-response-format

For swagger file:

https://github.com/adnan-kamili/swagger-response-template

Answer 6

Interesting discussion regarding Designing REST API for returning count of multiple objects: https://groups.google.com/g/api-craft/c/qbI2QRrpFew/m/h30DYnrqEwAJ?pli=1

As an API consumer, I would expect each count value to be represented either as a subresource to the countable resource (i.e. GET /tasks/count for a count of tasks), or as a field in a bigger aggregation of metadata related to the concerned resource (i.e. GET /tasks/metadata). By scoping related endpoints under the same parent resource (i.e. /tasks), the API becomes intuitive, and the purpose of an endpoint can (usually) be inferred from its path and HTTP method.

Additional thoughts:

1. If each individual count is only useful in combination with other counts (for a statistics dashboard, for example), you could possibly expose a single endpoint which aggregates and returns all counts at once.
2. If you have an existing endpoint for listing all resources (i.e. GET /tasks for listing all tasks), the count could be included in the response as metadata, either as HTTP headers or in the response body. Doing this will incur unnecessary load on the API, which might be negligible depending on your use case.