Very Slow upload/download speed of Firebase Storage

Question

The Story

I am using Firebase Storage in my app to upload large files into the Firebase Storage. Files are mostly videos which can be even larger than 2

Answer 1

I offer this not to refute your observations, but just as a point of comparison. As a test, I used the code below to upload a ~50MB file and time the progress. The test was run on a Samsung Galaxy S3 (4.4.2) with a home WiFi connection (in California). The average throughput was roughly 760KB/sec.

I ran the same test on other phone devices using the same network with throughput ranging from 300KB/sec to 850KB/sec (Moto X Pure; Marshmallow).

private void uploadTest() {
    // This file is ~50MB
    final File file = new File("/storage/emulated/0/DCIM/Camera/20160821_101145.mp4");
    // Upload the file
    Log.i(TAG, String.format("uploadTest: Starting upload of %5.2fMB file",
            file.length()/1024.0/1024.0));

    FirebaseStorage.getInstance().getReference("test").putFile(Uri.fromFile(file))
            .addOnSuccessListener(new OnSuccessListener<UploadTask.TaskSnapshot>() {
                @Override
                public void onSuccess(UploadTask.TaskSnapshot taskSnapshot) {
                    Log.i(TAG, "onSuccess: Done");
                }
            }).addOnProgressListener(new OnProgressListener<UploadTask.TaskSnapshot>() {
        @Override
        public void onProgress(UploadTask.TaskSnapshot taskSnapshot) {
            Log.i(TAG, String.format("onProgress: %5.2f MB transferred",
                    taskSnapshot.getBytesTransferred()/1024.0/1024.0));
        }
    });
}

This is the logcat produced, with some progress lines deleted to make the list more compact:

08-21 11:39:17.539 18427-18427/com.qbix.test I/MainActivity: uploadTest: Starting upload of 51.79MB file
08-21 11:39:17.559 18427-18427/com.qbix.test I/MainActivity: onProgress:  0.00 MB transferred
08-21 11:39:25.117 18427-18427/com.qbix.test I/MainActivity: onProgress:  5.00 MB transferred
08-21 11:39:31.654 18427-18427/com.qbix.test I/MainActivity: onProgress: 10.00 MB transferred
08-21 11:39:38.711 18427-18427/com.qbix.test I/MainActivity: onProgress: 15.00 MB transferred
08-21 11:39:45.088 18427-18427/com.qbix.test I/MainActivity: onProgress: 20.00 MB transferred
08-21 11:39:51.375 18427-18427/com.qbix.test I/MainActivity: onProgress: 25.00 MB transferred
08-21 11:39:57.411 18427-18427/com.qbix.test I/MainActivity: onProgress: 30.00 MB transferred
08-21 11:40:03.408 18427-18427/com.qbix.test I/MainActivity: onProgress: 35.00 MB transferred
08-21 11:40:10.886 18427-18427/com.qbix.test I/MainActivity: onProgress: 40.00 MB transferred
08-21 11:40:17.233 18427-18427/com.qbix.test I/MainActivity: onProgress: 45.00 MB transferred
08-21 11:40:23.069 18427-18427/com.qbix.test I/MainActivity: onProgress: 50.00 MB transferred
08-21 11:40:25.792 18427-18427/com.qbix.test I/MainActivity: onProgress: 51.79 MB transferred
08-21 11:40:25.792 18427-18427/com.qbix.test I/MainActivity: onSuccess: Done 

Answer 2

I've been doing some testing on my end and I'm trying to get to the bottom of why you see bad perf. I have been able to replicate bad performance when using getStream doing reads of single bytes at a time but this can be fixed by doing a read into a reasonably size byte buffer (256kb buffer). After fixing this, the performance I see is equal to downloading directly with httpsUrlConnection.

Three possibilities pop into my mind:

1) you are seeing timeouts during dns resolution (to our multiple ips) and connection pooling isnt working during uploads. But this would only explain why uploads (which are chunked) would be slow. Downloads are not chunked.

2) you are in a region we have not yet optimized for. We are rolling out local support soon across asia and europe to optimized performance.

3) Maybe you are comparing it with something other than httpsurlconnection? Can you give more details of your test?

Here is my test:

        final long startTime = System.currentTimeMillis();
        // Test via Firebase
        mStorage.child(downloadPath).getStream(new StreamDownloadTask.StreamProcessor() {
               @Override
               public void doInBackground(StreamDownloadTask.TaskSnapshot taskSnapshot,
                      final InputStream inputStream) throws IOException {
                 readStream("FB", inputStream, startTime);
               }
        });

        //Test with AsyncTask
        AsyncTask<Void, Void, Void> task = new AsyncTask<Void, Void, Void>() {
            @Override
            protected Void doInBackground(Void... voids) {
                URL url = null;
                try {
                    url = new URL(uri.toString());
                } catch (MalformedURLException e) {
                    e.printStackTrace();
                }
                try {
                    HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
                    connection.connect();
                    if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
                        return null;
                    }
                    InputStream inputStream = connection.getInputStream();
                    readStream("ASYNC_TASK", inputStream, startTime);
                    inputStream.close();

                }
                catch(IOException e) {

                }

                return null;
            }
        };

        task.execute();

    private void readStream(String desc, InputStream inputStream, long startTime) {
      long bytesRead = 0;
      byte[] buffer = new byte[256*1024];
      int b = 1;
      try {
        while (b > 0) {
            b = inputStream.read(buffer);
            bytesRead += b;
                Log.i("giug", "PROGRESS_" + desc + ": bytesperMS:" +
                        (bytesRead / (System.currentTimeMillis() - startTime)));
        }
        inputStream.close();
      } catch (IOException e) {
          e.printStackTrace();
      }
   }