Athena query fails with boto3 (S3 location invalid)

问题

I'm trying to execute a query in Athena, but it fails.

Code:

client.start_query_execution(QueryString="CREATE DATABASE IF NOT EXISTS db;",
                           QueryExecutionContext={'Database': 'db'},
                           ResultConfiguration={
                                     'OutputLocation': "s3://my-bucket/",
                                     'EncryptionConfiguration': {
                                             'EncryptionOption': 'SSE-S3'
                                             }
                                     })

But it raises the following exception:

botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) 
when calling the StartQueryExecution operation: The S3 location provided to save your 
query results is invalid. Please check your S3 location is correct and is in the same 
region and try again. If you continue to see the issue, contact customer support 
for further assistance.

However, if I go to the Athena Console, go to Settings and enter the same S3 location (for example):

the query runs fine.

What's wrong with my code? I've used the API of several the other services (eg, S3) successfully, but in this one I believe I'm passing some incorrect parameters. Thanks.

Python: 3.6.1. Boto3: 1.4.4

回答1:

EDIT: As suggested by Justin, AWS later added support for Athena by adding athena prefix to the bucket. Please upvote his answer.

---

Accepted Answer:

The S3 location provided to save your query results is invalid. Please check your S3 location is correct and is in the same region and try again.

Since it works when you use the console, it is likely the bucket is in a different region than the one you are using in Boto3. Make sure you use the correct region (the one that worked in the console) when constructing the Boto3 client. By default, Boto3 will use the region configured in the credentials file.

回答2:

I had to add a 'athena-' prefix to my bucket to get it to work. For example, in stead of:

"s3://my-bucket/"

Try:

"s3://athena-my-bucket/"

回答3:

Alternatively try boto3.client('athena', region_name = '<region>')

Ran into the same issue and needed to specify the S3 bucket in the client.

回答4:

In my case, IAM role didn't have all the permissions for the S3 bucket. I gave IAM role following permissions for Athena results bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::athena_results_bucket",
                "arn:aws:s3:::athena_results_bucket"
            ],
            "Effect": "Allow"
        }
    ]
}

来源：https://stackoverflow.com/questions/45313763/athena-query-fails-with-boto3-s3-location-invalid