Fast way of deleting non-empty Google bucket?

Question

Is this my only option or is there a faster way?

# Delete contents in bucket (takes a long time on large bucket)
gsutil -m rm -r gs://my-bucket/*

# Remove bucke         


        

Answer 1

Use this to set an appropriate lifecycle rule. e.g. wait for a day.

https://cloud.google.com/storage/docs/gsutil/commands/lifecycle

Example (Read carefully before copy paste)

gsutil lifecycle set [LIFECYCLE_CONFIG_FILE] gs://[BUCKET_NAME]

Example (Read carefully before copy paste)

{
  "rule":
  [
    {
      "action": {"type": "Delete"},
      "condition": {"age": 1}
    }
  ]
}

Then delete the bucket.

This will delete the data asynchronously, so you don't have to keep some background job running on your end.

Answer 2

Using Python client, you can force a delete within your script by using:

bucket.delete(force=True)

Try out a similar thing in your current language.

Github thread that discusses this

Answer 3

This deserves to be summarized and pointed out.

Deleting with gsutil rm is slow if you have LOTS (terabytes) of data

gsutil -m rm -r gs://my-bucket

However, you can specify the expiration for the bucket and let the GCS do the work for you. Create a fast-delete.json policy:

{
   "rule":[
      {
         "action":{
            "type":"Delete"
         },
         "condition":{
            "age":0
         }
      }
   ]
}

then apply

gsutil lifecycle set fast-delete.json gs://MY-BUCKET

Thanks, @jterrace and @Janosch

Answer 4

Buckets are required to be empty before they're deleted. So before you can delete a bucket, you have to delete all of the objects it contains.

You can do this with gsutil rm -r (documentation). Just don't pass the * wildcard and it will delete the bucket itself after it has deleted all of the objects.

gsutil -m rm -r gs://my-bucket

Google Cloud Storage bucket deletes can't succeed until the bucket listing returns 0 objects. If objects remain, you can get a Bucket Not Empty error (or in the UI's case 'Bucket Not Ready') when trying to delete the bucket.

gsutil has built-in retry logic to delete both buckets and objects.

Answer 5

Shorter one liner for the lifecycle change:

gsutil lifecycle set <(echo '{"rule":[{"action":{"type":"Delete"},"condition":{"age":0}}]}') gs://MY-BUCKET

I've also had good luck creating an empty bucket then starting a transfer to the bucket I want to empty out. Our largest bucket took about an hour to empty this way; the lifecycle method seems to take at least a day.

Answer 6

Remove the bucket from Developers Console. It will ask for confirmation before deleting a non empty bucket. It works like a charm ;)