Listing just the sub folders in an s3 bucket

Question

I have an s3 structure as follows:

s3bucketname -> List of first level keys -> List of second level keys -> List of third level keys -> Actual fi         


        

Answer 1

Charles version is super concise! thanks @charles-menguy

I wrote an extension to support huge list through pagination.

    public List<String> getSubPathsInS3Prefix(String bucketName, String prefix) {
        if (!prefix.endsWith(FILE_DELIMITER)) {
            prefix += FILE_DELIMITER;
        }
        List<String> paths = new ArrayList<String>();
        ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName(bucketName).withPrefix(prefix)
                .withMaxKeys(1000).withDelimiter(FILE_DELIMITER);
        ObjectListing currentListing = s3Client.listObjects(listObjectsRequest);
        paths.addAll(currentListing.getCommonPrefixes());

        while (currentListing == null || currentListing.isTruncated()) {
            currentListing = s3Client.listNextBatchOfObjects(currentListing);
            paths.addAll(currentListing.getCommonPrefixes());
        }
        return paths;
    }

http://www.lazywiz.com/uncategorized/s3-missing-api-list-sub-paths-in-the-s3-bucket/

Answer 2

I did the following code which seems to work fine, you have to pass a prefix and make sure the prefix ends with /, and also specify the delimiter you want to get your list of sub-directories. The following should work:

public List<String> listKeysInDirectory(String bucketName, String prefix) {
    String delimiter = "/";
    if (!prefix.endsWith(delimiter)) {
        prefix += delimiter;
    }

    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
            .withBucketName(bucketName).withPrefix(prefix)
            .withDelimiter(delimiter);
    ObjectListing objects = _client.listObjects(listObjectsRequest);
    return objects.getCommonPrefixes();
}