Batch Uploading Huge Sets of Images to Azure Blob Storage

Question

I have about 110,000 images of various formats (jpg, png and gif) and sizes (2-40KB) stored locally on my hard drive. I need to upload them to Azure Blob Storage. While doing th

Answer 1

You should definitely upload in parallel in several streams (ie. post multiple files concurrently), but before you do any experiment showing (erroneously) that there is not benefit, make sure you actually increase the value of ServicePointManager.DefaultConnectionLimit:

The maximum number of concurrent connections allowed by a ServicePoint object. The default value is 2.

With a default value of 2, you can have at most two outstanding HTTP requests against any destination.