How to delete all datastore in Google App Engine?

Question

Does anyone know how to delete all datastore in Google App Engine?

Answer 1

For all people that need a quick solution for the dev server (as time of writing in Feb. 2016):

1. Stop the dev server.
2. Delete the target directory.
3. Rebuild the project.

This will wipe all data from the datastore.

Answer 2

The zero-setup way to do this is to send an execute-arbitrary-code HTTP request to the admin service that your running app already, automatically, has:

import urllib
import urllib2

urllib2.urlopen('http://localhost:8080/_ah/admin/interactive/execute',
    data = urllib.urlencode({'code' : 'from google.appengine.ext import db\n' +
                                      'db.delete(db.Query())'}))

Answer 3

This is what you're looking for...

db.delete(Entry.all(keys_only=True))

Running a keys-only query is much faster than a full fetch, and your quota will take a smaller hit because keys-only queries are considered small ops.

Here's a link to an answer from Nick Johnson describing it further.

Below is an end-to-end REST API solution to truncating a table...

I setup a REST API to handle database transactions where routes are directly mapped through to the proper model/action. This can be called by entering the right url (example.com/inventory/truncate) and logging in.

Here's the route:

Route('/inventory/truncate', DataHandler, defaults={'_model':'Inventory', '_action':'truncate'})

Here's the handler:

class DataHandler(webapp2.RequestHandler):
  @basic_auth
  def delete(self, **defaults):
    model = defaults.get('_model')
    action = defaults.get('_action')
    module = __import__('api.models', fromlist=[model])
    model_instance = getattr(module, model)()
    result = getattr(model_instance, action)()

It starts by loading the model dynamically (ie Inventory found under api.models), then calls the correct method (Inventory.truncate()) as specified in the action parameter.

The @basic_auth is a decorator/wrapper that provides authentication for sensitive operations (ie POST/DELETE). There's also an oAuth decorator available if you're concerned about security.

Finally, the action is called:

def truncate(self):
  db.delete(Inventory.all(keys_only=True))

It looks like magic but it's actually very straightforward. The best part is, delete() can be re-used to handle deleting one-or-many results by adding another action to the model.

Answer 4

Adding answer about recent developments.

Google recently added datastore admin feature. You can backup, delete or copy your entities to another app using this console.

https://developers.google.com/appengine/docs/adminconsole/datastoreadmin#Deleting_Entities_in_Bulk

Answer 5

I often don't want to delete all the data store so I pull a clean copy of /war/WEB-INF/local_db.bin out source control. It may just be me but it seems even with the Dev Mode stopped I have to physically remove the file before pulling it. This is on Windows using the subversion plugin for Eclipse.

Answer 6

For any datastore that's on app engine, rather than local, you can use the new Datastore API. Here's a primer for how to get started.

I wrote a script that deletes all non-built in entities. The API is changing pretty rapidly, so for reference, I cloned it at commit 990ab5c7f2063e8147bcc56ee222836fd3d6e15b

from gcloud import datastore
from gcloud.datastore import SCOPE
from gcloud.datastore.connection import Connection
from gcloud.datastore import query

from oauth2client import client

def get_connection():
  client_email = 'XXXXXXXX@developer.gserviceaccount.com'
  private_key_string = open('/path/to/yourfile.p12', 'rb').read()

  svc_account_credentials = client.SignedJwtAssertionCredentials(
    service_account_name=client_email,
    private_key=private_key_string,
    scope=SCOPE)

  return Connection(credentials=svc_account_credentials)


def connect_to_dataset(dataset_id):
  connection = get_connection()
  datastore.set_default_connection(connection)
  datastore.set_default_dataset_id(dataset_id)

if __name__ == "__main__":
  connect_to_dataset(DATASET_NAME)
  gae_entity_query = query.Query()
  gae_entity_query.keys_only()
  for entity in gae_entity_query.fetch():
    if entity.kind[0] != '_':
      print entity.kind
      entity.key.delete()