How much quota does an appengine datastore Query cost?

Question

In the appengine Billing and Budgeting Resources page, it says that the cost of a "Query" maps to "1 read + 1 small per entity retrieved", whereas a "Query (keys only)" maps to "1 read + 1 small per key retrieved".

This seems like a typo to me. It would seem that a Query would still need to perform a full "get" operation on each entity returned. Is this assumption incorrect? I would have expected the cost of a "Query" to be "1 read + 1 read per entity retrieved".

Answer 1

This definitely looks like a typo. cpm_usd looks like a deprecated way to measure costs, linked to the previous pricing model.

With a recent version of AppStats (Python SDK 1.7.1) there is a tool to compute datastore related costs. Using the interactive playground I quickly got these results:

• Query with keys_only=False

  @1ms datastore_v3.RunQuery real=36ms api=0ms cost=770 billed_ops=[DATASTORE_READ:11]

• Same Query with keys_only=True

  @1ms datastore_v3.RunQuery real=5ms api=0ms cost=170 billed_ops=[DATASTORE_READ:1, DATASTORE_SMALL:10]

(All costs displayed in micropennies (1 dollar equals 100 pennies, 1 penny equals 1 million micropennies))

Answer 2

Hmm, this does seem strange. I'd guess that keys only query only looks into indexes, while normal query also retrieves entity based on that key.

Anyhow, it's easy to test this: all requests have cost added to the log. Create one requests that performs a query and another with the same keys-only query and then compare the costs.

Answer 3

I just tested a keys-only query against a regular one as Peter suggested.

Here's the regular query:

def test_query():
  q = Project.all()
  q.run()

  return 'Query test complete.'

And the log:

70.162.229.226 - - [02/Sep/2012:20:46:51 -0700] "GET /query HTTP/1.1" 200 124 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1" "www.kicksaver.net" ms=28 cpu_ms=0 cpm_usd=0.000014 instance=00c61b117c79c7b82c3798e359e96ca71deb39

The keys-only query:

def test_key_query():
  q = Project.all()
  q.run(keys_only=True)

  return 'Keys only test complete.'

And the log:

70.162.229.226 - - [02/Sep/2012:20:46:56 -0700] "GET /keys_only HTTP/1.1" 200 128 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1" "www.kicksaver.net" ms=29 cpu_ms=0 cpm_usd=0.000014 instance=00c61b117c79c7b82c3798e359e96ca71deb39

Both return cpm_usd=0.000014. I tested across two different applications and with a few different batch sizes and limits, and the cpm_usd values were always equal or within 0.000001 of each other. It looks like the documentation is correct as written.