How to get a list of all files in Cloud Storage in a Firebase app?

Question

I\'m working on uploading images, everything works great, but I have 100 pictures and I would like to show all of them in my View, as I get the complete list of

Answer 1

Actually this is possible but only with a Google Cloud API instead one from Firebase. It's because a Firebase Storage is a Google Cloud Storage Bucket which can be reached easily with the Google Cloud APIs however you need to use OAuth for Authentication instead of the Firebase one's.

Answer 2

Extending Rosário Pereira Fernandes' answer, for a JavaScript solution:

1. Install firebase on your machine

npm install -g firebase-tools

2. On firebase init set JavaScript as default language
3. On the root folder of created project execute npm installs

   npm install --save firebase
   npm install @google-cloud/storage
   npm install @google-cloud/firestore
   ... <any other dependency needed>

4. Add non-default dependencies on your project like

    "firebase": "^6.3.3",
    "@google-cloud/storage": "^3.0.3"

functions/package.json

{
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint .",
    "serve": "firebase serve --only functions",
    "shell": "firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "10"
  },
  "dependencies": {
    "@google-cloud/storage": "^3.0.3",
    "firebase": "^6.3.3",
    "firebase-admin": "^8.0.0",
    "firebase-functions": "^3.1.0"
  },
  "devDependencies": {
    "eslint": "^5.12.0",
    "eslint-plugin-promise": "^4.0.1",
    "firebase-functions-test": "^0.1.6"
  },
  "private": true
}

5. Create sort of a listAll function

index.js

var serviceAccount = require("./key.json");
const functions = require('firebase-functions');

const images = require('./images.js');

var admin = require("firebase-admin");

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    databaseURL: "https://<my_project>.firebaseio.com"
});

const bucket = admin.storage().bucket('<my_bucket>.appspot.com')

exports.getImages = functions.https.onRequest((request, response) => {
    images.getImages(bucket)
        .then(urls => response.status(200).send({ data: { urls } }))
        .catch(err => console.error(err));
})

images.js

module.exports = {
    getImages
}

const query = {
    directory: 'images'
};

function getImages(bucket) {
    return bucket.getFiles(query)
        .then(response => getUrls(response))
        .catch(err => console.error(err));
}

function getUrls(response) {
    const promises = []
    response.forEach( files => {
        files.forEach (file => {
            promises.push(getSignedUrl(file));
        });
    });
    return Promise.all(promises).then(result => getParsedUrls(result));
}

function getSignedUrl(file) {
    return file.getSignedUrl({
        action: 'read',
        expires: '09-01-2019'
    })
}

function getParsedUrls(result) {
    return JSON.stringify(result.map(mediaLink => createMedia(mediaLink)));
}

function createMedia(mediaLink) {
    const reference = {};
    reference.mediaLink = mediaLink[0];
    return reference;
}

6. Execute firebase deploy to upload your cloud function
7. Call your custom function from your app

build.gradle

dependencies {
...
  implementation 'com.google.firebase:firebase-functions:18.1.0'
...
}

kotlin class

  private val functions = FirebaseFunctions.getInstance()
  val cloudFunction = functions.getHttpsCallable("getImages")
  cloudFunction.call().addOnSuccessListener {...}

Regarding the further development of this feature, I ran into some problems that might found here.

Answer 3

Since Mar 2017: With the addition of Firebase Cloud Functions, and Firebase's deeper integration with Google Cloud, this is now possible.

With Cloud Functions you can use the Google Cloud Node package to do epic operations on Cloud Storage. Below is an example that gets all the file URLs into an array from Cloud Storage. This function will be triggered every time something's saved to google cloud storage.

Note 1: This is a rather computationally expensive operation, as it has to cycle through all files in a bucket / folder.

Note 2: I wrote this just as an example, without paying much detail into promises etc. Just to give an idea.

const functions = require('firebase-functions');
const gcs = require('@google-cloud/storage')();

// let's trigger this function with a file upload to google cloud storage

exports.fileUploaded = functions.storage.object().onChange(event => {

  const object = event.data; // the object that was just uploaded
  const bucket = gcs.bucket(object.bucket);
  const signedUrlConfig = { action: 'read', expires: '03-17-2025' }; // this is a signed url configuration object

  var fileURLs = []; // array to hold all file urls 

  // this is just for the sake of this example. Ideally you should get the path from the object that is uploaded :)
  const folderPath = "a/path/you/want/its/folder/size/calculated";

  bucket.getFiles({ prefix: folderPath }, function(err, files) {
    // files = array of file objects
    // not the contents of these files, we're not downloading the files. 

    files.forEach(function(file) {
      file.getSignedUrl(signedUrlConfig, function(err, fileURL) {
        console.log(fileURL);
        fileURLs.push(fileURL);
      });
    });

  });

});

I hope this will give you the general idea. For better cloud functions examples, check out Google's Github repo full of Cloud Functions samples for Firebase. Also check out their Google Cloud Node API Documentation

Answer 4

Since there's no language listed, I'll answer this in Swift. We highly recommend using Firebase Storage and the Firebase Realtime Database together to accomplish lists of downloads:

Shared:

// Firebase services
var database: FIRDatabase!
var storage: FIRStorage!
...
// Initialize Database, Auth, Storage
database = FIRDatabase.database()
storage = FIRStorage.storage()
...
// Initialize an array for your pictures
var picArray: [UIImage]()

Upload:

let fileData = NSData() // get data...
let storageRef = storage.reference().child("myFiles/myFile")
storageRef.putData(fileData).observeStatus(.Success) { (snapshot) in
  // When the image has successfully uploaded, we get it's download URL
  let downloadURL = snapshot.metadata?.downloadURL()?.absoluteString
  // Write the download URL to the Realtime Database
  let dbRef = database.reference().child("myFiles/myFile")
  dbRef.setValue(downloadURL)
}

Download:

let dbRef = database.reference().child("myFiles")
dbRef.observeEventType(.ChildAdded, withBlock: { (snapshot) in
  // Get download URL from snapshot
  let downloadURL = snapshot.value() as! String
  // Create a storage reference from the URL
  let storageRef = storage.referenceFromURL(downloadURL)
  // Download the data, assuming a max size of 1MB (you can change this as necessary)
  storageRef.dataWithMaxSize(1 * 1024 * 1024) { (data, error) -> Void in
    // Create a UIImage, add it to the array
    let pic = UIImage(data: data)
    picArray.append(pic)
  })
})

For more information, see Zero to App: Develop with Firebase, and it's associated source code, for a practical example of how to do this.

Answer 5

Since Firebase SDKs for JavaScript release 6.1, iOS release 6.4, and Android release version 18.1 all have a method to list files.

The documentation is a bit sparse so far, so I recommend checking out Rosário's answer for details.

---

Previous answer, since this approach can still be useful at times:

There currently is no API call in the Firebase SDK to list all files in a Cloud Storage folder from within an app. If you need such functionality, you should store the metadata of the files (such as the download URLs) in a place where you can list them. The Firebase Realtime Database and Cloud Firestore are perfect for this and allows you to also easily share the URLs with others.

You can find a good (but somewhat involved) sample of this in our FriendlyPix sample app. The relevant code for the web version is here, but there are also versions for iOS and Android.

Answer 6

Combining some answers from this post and also from here, and after some personal research, for NodeJS with typescript I managed to accomplish this by using firebase-admin:

import * as admin from 'firebase-admin';
const getFileNames = (folderName: any) => {
  admin.storage().bucket().getFiles(autoPaginate: false).then(([files]: any) => {
    const fileNames = files.map((file: any) => file.name);
    return fileNames;
  })
 }

In my case I also needed to get all the files from a specific folder from firebase storage. According to google storage the folders don't exists but are rather a naming conventions. Anyway I managed to to this by adding { prefix: ${folderName}, autoPaginate: false } the the getFiles function so:

getFiles({ prefix: `${folderName}`, autoPaginate: false })