How can I easily determine if a Boto 3 S3 bucket resource exists?

Question

For example, I have this code:

import boto3

s3 = boto3.resource(\'s3\')

bucket = s3.Bucket(\'my-bucket-name\')

# Does it exist???

Answer 1

you can use conn.get_bucket

from boto.s3.connection import S3Connection
from boto.exception import S3ResponseError    

conn = S3Connection(aws_access_key, aws_secret_key)

try:
    bucket = conn.get_bucket(unique_bucket_name, validate=True)
except S3ResponseError:
    bucket = conn.create_bucket(unique_bucket_name)

quoting the documentation at http://boto.readthedocs.org/en/latest/s3_tut.html

As of Boto v2.25.0, this now performs a HEAD request (less expensive but worse error messages).

Answer 2

At the time of this writing there is no high-level way to quickly check whether a bucket exists and you have access to it, but you can make a low-level call to the HeadBucket operation. This is the most inexpensive way to do this check:

from botocore.client import ClientError

try:
    s3.meta.client.head_bucket(Bucket=bucket.name)
except ClientError:
    # The bucket does not exist or you have no access.

Alternatively, you can also call create_bucket repeatedly. The operation is idempotent, so it will either create or just return the existing bucket, which is useful if you are checking existence to know whether you should create the bucket:

bucket = s3.create_bucket(Bucket='my-bucket-name')

As always, be sure to check out the official documentation.

Note: Before the 0.0.7 release, meta was a Python dictionary.

Answer 3

As mentioned by @Daniel, the best way as suggested by Boto3 docs is to use head_bucket()

head_bucket() - This operation is useful to determine if a bucket exists and you have permission to access it.

If you have a small number of buckets, you can use the following:

>>> import boto3
>>> s3 = boto3.resource('s3')
>>> s3.Bucket('Hello') in s3.buckets.all()
False
>>> s3.Bucket('some-docs') in s3.buckets.all()
True
>>> 

Answer 4

Use lookup Function -> Returns None if bucket Exist

if s3.lookup(bucketName) is None:
    bucket=s3.create_bucket(bucketName) # Bucket Don't Exist
else:
    bucket = s3.get_bucket(bucketName) #Bucket Exist

Answer 5

I've had success with this:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket-name')

if bucket.creation_date:
   print("The bucket exists")
else:
   print("The bucket does not exist")

Answer 6

I tried Daniel's example and it was really helpful. Followed up the boto3 documentation and here is my clean test code. I have added a check for '403' error when buckets are private and return a 'Forbidden!' error.

import boto3, botocore
s3 = boto3.resource('s3')
bucket_name = 'some-private-bucket'
#bucket_name = 'bucket-to-check'

bucket = s3.Bucket(bucket_name)
def check_bucket(bucket):
    try:
        s3.meta.client.head_bucket(Bucket=bucket_name)
        print("Bucket Exists!")
        return True
    except botocore.exceptions.ClientError as e:
        # If a client error is thrown, then check that it was a 404 error.
        # If it was a 404 error, then the bucket does not exist.
        error_code = int(e.response['Error']['Code'])
        if error_code == 403:
            print("Private Bucket. Forbidden Access!")
            return True
        elif error_code == 404:
            print("Bucket Does Not Exist!")
            return False

check_bucket(bucket)

Hope this helps some new into boto3 like me.