Start-AzureStorageBlobCopy vs AzCopy: which one takes lesser time

Question

I need to move vhds from one subscription to other. I would like to know which one is better option for the same: Start-AzureStorageBlobCopy or AzCopy

Answer 1

In most scenarios, AzCopy is likely to be quicker than Start-AzureStorageBlobCopy due to way you would initiate the copy resulting in fewer calls to Azure API:

• [AzCopy]1 call for whole container (regardless of blob count)

vs

• [Start-AzureStorageBlobCopy] N number of calls due to number of blobs in container.

Initially I thought it would be same as both appear to trigger same asynchronous copies on Azure side, however on client side this would be directly visible as @Evgeniy has found in his answer.

In 1 blob in container scenario, theoretically both commands would complete at same time.

*EDIT (possible workaround): I was able to decrease my time tremendously by:

1. Removing console output AND
2. Using the -ConcurrentTaskCount switch, set to 100 in my case. Cut it down to under 5 minutes now.

Answer 2

They don't take the same time.

I've tried to copy from one account to another and have a huge difference.

Start-AzureStorageBlobCopy -SrcBlob $_.Name -SrcContainer $Container -Context $ContextSrc -DestContainer $Container -DestBlob $_.Name -DestContext $ContextDst --Verbose

This takes about 2.5 hours.

& .\AzCopy.exe /Source:https://$StorageAccountNameSrc.blob.core.windows.net/$Container /Dest:https://$StorageAccountNameDst.blob.core.windows.net/$Container /SourceKey:$StorageAccountKeySrc /DestKey:$StorageAccountKeyDst /S

This takes several minutes.

I have about 600 Mb and about 7000 files here.

Elapsed time:            00.00:03:41
Finished 44 of total 44 file(s).
[2017/06/22 17:05:35] Transfer summary:
-----------------
Total files transferred: 44
Transfer successfully:   44
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:08
Finished 345 of total 345 file(s).
[2017/06/22 17:06:07] Transfer summary:
-----------------
Total files transferred: 345
Transfer successfully:   345
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:31

Do anyone know why it's so different?

Answer 3

Both of them would take the same time as all they do is initiate Async Server-Side Blob Copy. They just tell the service to start copying blob from source to destination. The actual copy operation is performed by Azure Blob Storage Service. The time it would take to copy the blob would depend on a number of factors including but not limited to:

• Source & destination location.
• Size of the source blob.
• Load on storage service.

Answer 4

AzCopy offers an SLA which the Async copy services lacks. AzCopy is designed for optimal performance. Use the/SyncCopy parameter to get a consistent copy speed.

Answer 5

Running AzCopy without specifying the option /SyncCopy and running PowerShell command Start-AzureStorageBlobCopy should take the same duration, because they both use server side asynchronous copy.

If you'd like to copy blobs across regions, you'd better consider specifying the option /SyncCopy while executing AzCopy in order to achieve a consistent speed because the asynchronous copying of data will run in the background of servers that being said you might see inconsistent copying speed among your “copying” operations.

If /SyncCopy option is specified, AzCopy will download the content to memory first, and then upload content back to Azure Storage. In order to achieve better performance of /SyncCopy, you are supposed to run AzCopy in the VM whose region is the same as source storage account. Besides that, the VM size (which decides bandwidth and CPU core number) will probably impact the copying performance as well.

For further information, please refer to Getting Started with the AzCopy Command-Line Utility