boto3 - AWS lambda -copy files between buckets

问题

I am trying to copy multiple files in a source bucket to a destination bucket using AWS lambda and am getting the error below. Bucket structures are as follows

Source Buckets

mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_0.csv.gz mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_1.csv.gz mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_count_20170926.inf

Destination Buckets

mydestbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_0.csv.gz

mydestbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_1.csv.gz mydestbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_count_20170926.inf

I wrote the lambda function below but am getting the error below. Can someone help me explain what I am doing wrong

{   "errorMessage": "expected string or bytes-like object",   "errorType": "TypeError",   "stackTrace": [
    [
      "/var/task/lambda_function.py",
      17,
      "lambda_handler",
      "s3.Object(dest_bucket,dest_key).copy_from(CopySource= { 'Bucket': obj.bucket_name , 'Key' : obj.key})"
    ],
    [
      "/var/runtime/boto3/resources/factory.py",
      520,
      "do_action",
      "response = action(self, *args, **kwargs)"
    ],
    [
      "/var/runtime/boto3/resources/action.py",
      83,
      "__call__",
      "response = getattr(parent.meta.client, operation_name)(**params)"
    ],
    [
      "/var/runtime/botocore/client.py",
      312,
      "_api_call",
      "return self._make_api_call(operation_name, kwargs)"
    ],
    [
      "/var/runtime/botocore/client.py",
      575,
      "_make_api_call",
      "api_params, operation_model, context=request_context)"
    ],
    [
      "/var/runtime/botocore/client.py",
      627,
      "_convert_to_request_dict",
      "params=api_params, model=operation_model, context=context)"
    ],
    [
      "/var/runtime/botocore/hooks.py",
      227,
      "emit",
      "return self._emit(event_name, kwargs)"
    ],
    [
      "/var/runtime/botocore/hooks.py",
      210,
      "_emit",
      "response = handler(**kwargs)"
    ],
    [
      "/var/runtime/botocore/handlers.py",
      208,
      "validate_bucket_name",
      "if VALID_BUCKET.search(bucket) is None:"
    ]   ] }

Lambda Function Code

import boto3
import json
s3 = boto3.resource('s3')



def lambda_handler (event, context):
 bucket = s3.Bucket('mysrcbucket')
 dest_bucket=s3.Bucket('mydestbucket')
 print(bucket)
 print(dest_bucket)

for obj in bucket.objects.filter(Prefix='Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF',Delimiter='/'):

  dest_key=obj.key
  print(dest_key)
  s3.Object(dest_bucket,dest_key).copy_from(CopySource= { 'Bucket': obj.bucket_name , 'Key' : obj.key})

回答1:

The issue is with:

s3.Object(dest_bucket, dest_key).copy_from(CopySource= {'Bucket': obj.bucket_name, 
                                                        'Key': obj.key})

change dest_bucket to dest_bucket.name:

s3.Object(dest_bucket.name, dest_key).copy_from(CopySource= {'Bucket': obj.bucket_name,
                                                             'Key': obj.key})

dest_bucket is a resource and name is its identifier.

来源：https://stackoverflow.com/questions/46434082/boto3-aws-lambda-copy-files-between-buckets

标签

python-3.x

amazon-web-services

amazon-s3

boto3