Amazon s3 Folders Problem

Question

I need off help regarding amazon s3 with folders,

The problem i get with the Amazon s3 class by undesigned is it doesnt support folders, it will only show you the fu

Answer 1

Amazon S3 is a flat file system. There is no such thing as folders. There are simply really long file names with slashes in them. Most S3 tools will visually display these as folders, but that's not how S3 works under the hood.

I've never used the Undesigned Amazon S3 class before, so I'm not sure about the specifics of that library. I used CloudFusion's S3 class for a long time until Amazon forked it to create the official PHP SDK (which is what I use now).

In the PHP SDK, you can do:

$s3 = new AmazonS3();

$response = $s3->list_objects($bucket, array(
    'prefix' => 'Music/dnb/'
));

print_r($response->body);

That will list all objects in your S3 bucket that have a file name (no real folders, remember?) that begins with Music/dnb/.

Answer 2

There is no way around that in the Amazon API. You need to filter your amazon request using the prefix as you are and then filter out the 'subfolders' on the client.

The best solution actually, is not to try to navigate your S3 storage. Instead you should maintain an 'index' of your files in a proper database (MySql, SimpleDb etc) that you search and query against, and then just retrieve the file from S3 when needed.

Answer 3

I don´t know if this response it will be usefull after so long, but here we go:

To resolve this problem you have to do this in java:

            List<S3ObjectSummary> files = s3client.listObjects(bucket.getName()).getObjectSummaries();
            for (S3ObjectSummary file : files) {
                if (file.getKey().endsWith("/"))
                    System.out.println("----" + file.getKey() + " (ES CARPETA)");
                else 
                    System.out.println("----" + file.getKey() + " NO NO NO");
            }

With the method "endsWith("/")" you can detect if the S3ObjectSummary is a folder or not.

Hope this can helps someone.

Mike

Answer 4

$contents = $s3->getBucket("MY-BUCKET", $amazon_folder, null, null, "/");

Answer 5

So, Here's the work around. To list your directories you do this.

   `val objRequest = new ListObjectsRequest()
        .withBucketName("MY-BUCKET")
        .withPrefix("Music/").withDelimiter("/")
    val result     = s3Client.listObjects(objRequest)
    result.getCommonPrefixes()`

This will list all the folders in MYBUCKET/Music.

This is a away you get aroung with having a flat file store. (This is written in scala)