Python: Amazon S3 cannot get the bucket: says 403 Forbidden

Question

I have a bucket for my organization in Amazon S3 which looks like mydev.orgname

• I have a Java application that can connect to Amazon S3 with th

Answer 1

After giving my "User" much stronger role, this error was gone. Means User given the permission to get_bucket

Answer 2

this answer work for me :)

I did

• S3 bucket policy setting
• time setting
• bucket = conn.get_bucket(BUCKET_NAME, validate=False)

Answer 3

Giving the user a "stronger role" is not the correct solution. This is simply a problem with boto library usage. Clearly, you don't need extra permissions when using Java S3 library.

Correct way to use boto in this case is:

b = conn.get_bucket('my-bucket', validate=False)
k = b.get_key('my/cool/object.txt') # will send HEAD request to S3
...

Basically, boto by default (which is a mistake on their part IMHO), assumes you want to interact with S3 bucket. Granted, sometimes you do want that, but then you should use credentials that have permissions for S3 bucket operations. But a more popular use case is to interact with S3 objects, and in this case you don't need any special bucket-level permissions, hence the use of validate=False kwarg.

Answer 4

Read files from Amazon S3 bucket using Python

import boto3
import csv

# get a handle on s3
session = boto3.Session(
                aws_access_key_id='XXXXXXXXXXXXXXXXXXXXXXX',
                aws_secret_access_key='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
                region_name='XXXXXXXXXX')

s3 = session.resource('s3')

# get a handle on the bucket that holds your file
bucket = s3.Bucket('bucket name') # example: energy_market_procesing

# get a handle on the object you want (i.e. your file)
obj = bucket.Object(key='file to read') # example: market/zone1/data.csv

# get the object
response = obj.get()

# read the contents of the file
lines = response['Body'].read()

# saving the file data in a new file test.csv
with open('test.csv', 'wb') as file:
    file.write(lines)