How to delete all datastore in Google App Engine?

Question

Does anyone know how to delete all datastore in Google App Engine?

Answer 1

There are several ways you can use to remove entries from App Engine's Datastore:

1. First, think whether you really need to remove entries. This is expensive and it might be cheaper to not remove them.

2. You can delete all entries by hand using the Datastore Admin.

3. You can use the Remote API and remove entries interactively.

4. You can remove the entries programmatically using a couple lines of code.

5. You can remove them in bulk using Task Queues and Cursors.

6. Or you can use Mapreduce to get something more robust and fancier.

Each one of these methods is explained in the following blog post: http://www.shiftedup.com/2015/03/28/how-to-bulk-delete-entries-in-app-engine-datastore

Hope it helps!

Answer 2

If you're talking about the live datastore, open the dashboard for your app (login on appengine) then datastore --> dataviewer, select all the rows for the table you want to delete and hit the delete button (you'll have to do this for all your tables). You can do the same programmatically through the remote_api (but I never used it).

If you're talking about the development datastore, you'll just have to delete the following file: "./WEB-INF/appengine-generated/local_db.bin". The file will be generated for you again next time you run the development server and you'll have a clear db.

Make sure to clean your project afterwards.

This is one of the little gotchas that come in handy when you start playing with the Google Application Engine. You'll find yourself persisting objects into the datastore then changing the JDO object model for your persistable entities ending up with obsolete data that'll make your app crash all over the place.

Answer 3

For the development server, instead of running the server through the google app engine launcher, you can run it from the terminal like:

dev_appserver.py --port=[portnumber] --clear_datastore=yes [nameofapplication]

ex: my application "reader" runs on port 15080. After modify the code and restart the server, I just run "dev_appserver.py --port=15080 --clear_datastore=yes reader".

It's good for me.

Answer 4

The best approach is the remote API method as suggested by Nick, he's an App Engine engineer from Google, so trust him.

It's not that difficult to do, and the latest 1.2.5 SDK provides the remote_shell_api.py out of the shelf. So go to download the new SDK. Then follow the steps:

• connect remote server in your commandline: remote_shell_api.py yourapp /remote_api The shell will ask for your login info, and if authorized, will make a Python shell for you. You need setup url handler for /remote_api in your app.yaml

• fetch the entities you'd like to delete, the code looks something like:

    from models import Entry
    query = Entry.all(keys_only=True)
    entries =query.fetch(1000)
    db.delete(entries)
    \# This could bulk delete 1000 entities a time

Update 2013-10-28:

• remote_shell_api.py has been replaced by remote_api_shell.py, and you should connect with remote_api_shell.py -s your_app_id.appspot.com, according to the documentation.

• There is a new experimental feature Datastore Admin, after enabling it in app settings, you can bulk delete as well as backup your datastore through the web ui.

Answer 5

PHP variation:

import com.google.appengine.api.datastore.Query;
import com.google.appengine.api.datastore.DatastoreServiceFactory;

define('DATASTORE_SERVICE', DatastoreServiceFactory::getDatastoreService());

function get_all($kind) {
    $query = new Query($kind);
    $prepared = DATASTORE_SERVICE->prepare($query);
    return $prepared->asIterable();
}

function delete_all($kind, $amount = 0) {
    if ($entities = get_all($kind)) {
        $r = $t = 0;
        $delete = array();
        foreach ($entities as $entity) {
            if ($r < 500) {
                $delete[] = $entity->getKey();
            } else {
                DATASTORE_SERVICE->delete($delete);
                $delete = array();
                $r = -1;
            }
            $r++; $t++;
            if ($amount && $amount < $t) break;
        }
        if ($delete) {
            DATASTORE_SERVICE->delete($delete);
        }
    }
}

Yes it will take time and 30 sec. is a limit. I'm thinking to put an ajax app sample to automate beyond 30 sec.

Answer 6

Open "Datastore Admin" for your application and enable Admin. Then all of your entities will be listed with check boxes. You can simply select the unwanted entites and delete them.