Reading part of a file in S3 using Boto

问题

I am trying to read 700MB file stored in S3. How ever I only require bytes from locations 73 to 1024.

I tried to find a usable solution but failed to. Would be a great help if someone could help me out.

回答1:

S3 supports GET requests using the 'Range' HTTP header which is what you're after.

To specify a Range request in boto, just add a header dictionary specifying the 'Range' key for the bytes you are interested in. Adapted from Mitchell Garnaat's response:

import boto
s3 = boto.connect_s3()
bucket = s3.lookup('mybucket')
key = bucket.lookup('mykey')
your_bytes = key.get_contents_as_string(headers={'Range' : 'bytes=73-1024'})

回答2:

import boto3

obj = boto3.resource('s3').Object('mybucket', 'mykey')
stream = obj.get(Range='bytes=32-64')['Body']
print(stream.read())

boto3 version from https://github.com/boto/boto3/issues/1236

回答3:

Please have a look on the python script here

import boto3
region = 'us-east-1' # define your region here
bucketname = 'test'  # define bucket
key = 'objkey' # s3 file 
Bytes_range = 'bytes=73-1024'
client = boto3.client('s3',region_name = region)
resp = client.get_object(Bucket=bucketname,Key=key,Range=Bytes_range)
data = resp['Body'].read()

来源：https://stackoverflow.com/questions/30075978/reading-part-of-a-file-in-s3-using-boto