Determine if folder or file key - Boto

Question

Using boto and Python I am trying to differentiate whether a key is returning a folder of file (I am aware that S3 treats both exactly the same as I am not dealing with a fi

Answer 1

You are correct that folders do not exist. You could, for example, create a file called output/2020/01/01/foo.txt even if none of those sub-folders exist.

Some systems, however, like to 'create' a folder by making a zero-length object with the name of the pretend folder. In such cases, you can identify the 'folder' by checking the length of the object.

Here's some sample code (using boto3 client method):

import boto3

s3 = boto3.client('s3', region_name = 'ap-southeast-2')

response = s3.list_objects_v2(Bucket='my-bucket')

for object in response['Contents']:
  if object['Size'] == 0:
      # Print name of zero-size object
      print(object['Key'])

Officially, there is no reason for such 'folder files' to exist. Amazon S3 will work perfectly well without them (and often better, for reasons you are discovering!).

Answer 2

S3.Object and S3.ObjectSummary will have the following property:

'ContentType': 'application/x-directory'

if the key is a directory.

for s3_obj_summary in bucket.objects.all():
  if s3_obj_summary.get()['ContentType'] == 'application/x-directory':
    print(str(s3_obj_summary))

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ObjectSummary.get