Rename an incoming S3 file with a random directory structure

问题

I have this app that will send a file to a s3 bucket. unfortunately I cannot change the path it sends it to in s3 so I have to figure out a way to get this file.

mys3bucket: /apps/region/020-07-14T22:24:34Z/details.csv

As you can see the date, the app places the date into the path. I am trying to not hard code items to make it more flexible.

what I want to do is get that details.csv file rename and move it to another location within the same s3 bucket. basically its permanent location.

what I was trying was something like this but it clearly will not work with the random path. the only piece that I can make a variable for is:

path = /apps/region/ the next level is random, but the report name is always the same.

clearly im not trying this the correct way but as of now I am not sure.

s3.Object( 'mys3bucket' ,'account3_details.csv').copy_from(CopySource='mys3bucket/apps/region/2020-07-14T22:24:34Z/details.csv')
s3.Object( 'mys3bucket','/apps/region/2020-07-14T22:24:34Z/details.csv').delete()

回答1:

Here is some code from a function I previously wrote. When triggered by an Amazon S3 Event, it moves the object to a new location:

import boto3
import urllib

TARGET_BUCKET = 'my-bucket'
TARGET_PATH = 'foo/'

def lambda_handler(event, context):
    
    # Get incoming bucket and key
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    source_key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])

    # Extract filename without path
    filename = ('/' + source_key).rsplit('/', 1)[1]

    # Copy object to different bucket
    s3_resource = boto3.resource('s3')
    copy_source = {
        'Bucket': source_bucket,
        'Key': source_key
    }
    s3_resource.Bucket(TARGET_BUCKET).Object(TARGET_PATH + filename).copy(copy_source)
    s3_resource.Bucket(source_bucket).Object(source_key).delete()

You would need to modify the logic that determines the destination filename (Key) of the copied object.

来源：https://stackoverflow.com/questions/62925258/rename-an-incoming-s3-file-with-a-random-directory-structure

标签

amazon-web-services

amazon-s3

boto3