How can I get only one level of objects in a S3 bucket?

Question

I want to list only the objects in a bucket that aren\'t buckets themselves. Is there a way of doing this short of parsing out the results of ListBucket?

Answer 1

There is a better approach using the latest boto3 version (1.14 as of now) and list_objects_v2 method.

import boto3

s3_client = boto3.client('s3')
response = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=your_prefix)

print(response)

Answer 2

objects in a bucket that aren't buckets themselves

Buckets can't contain other buckets. Do you mean folders? S3 doesn't have a concept of folders either.

You can have 100 buckets per S3 account and each bucket can contain an unlimited number of objects/files. If you name your files with /'s in the filename, the AWS GUI tools (eg AWS Console, BucketExplorer etc) will interpret each section as a virtual folder. eg

A file named folder1/folder2/myfile.jpg will be stored in S3 as a 'flat' file with that name, but in the GUI tools it will appear as though a file named myfile.jpg is 2 subfolders down in folder1/folder2.

You can use the prefix and delimiter parameters to parse the results of a GET Bucket (List Objects) call. The same options are available in any of the SDKs too.

UPDATE to answer comment.

Assuming our S3 bucket looks like this:

mybucket
   folder1
      file1.txt
      file2.txt
      folder2
          file3.txt
          file4.txt
      folder3
          file5.txt
          file6.txt

Using prefix = "folder1/" would return all 6 files : file1.txt to file6.txt.

Using a prefix = "folder1/" and a delimiter = "/" would return 2 files:

    file1.txt
    file2.txt

And the CommonPrefixes collection of the response with contain

    folder1/folder2/
    folder1/folder3/