AzCopy Sync command is failing

问题

I'm issuing this command:

azcopy sync "D:\Releases\Test\MyApp" "http://server3:10000/devstoreaccount1/myapp?sv=2019-02-02&st=2020-06-24T03%3A19%3A44Z&se=2020-06-25T03%3A19%3A44Z&sr=c&sp=racwdl&sig=REDACTED"

...and I'm getting this error:

error parsing the input given by the user. Failed with error Unable to infer the source 'D:\Releases\Test\MyApp' / destination 'http://server3:10000/devstoreaccount1/myapp?sv=2019-02-02&st=2020-06-24T03%3A19%3A44Z&se=2020-06-25T03%3A19%3A44Z&sr=c&sp=racwdl&sig=-REDACTED-

I would have thought my source was pretty clear.

Can anyone see anything wrong with my syntax?

回答1:

I believe you have run into an issue with azcopy that it does not support local emulator (at least for sync command). There's an open issue on Github for the same: https://github.com/Azure/azure-storage-azcopy/issues/554.

Basically the issue is coming from the following lines of code, where it returns location as Unknown in case of storage emulator URLs:

func inferArgumentLocation(arg string) common.Location {
    if arg == pipeLocation {
        return common.ELocation.Pipe()
    }
    if startsWith(arg, "http") {
        // Let's try to parse the argument as a URL
        u, err := url.Parse(arg)
        // NOTE: sometimes, a local path can also be parsed as a url. To avoid thinking it's a URL, check Scheme, Host, and Path
        if err == nil && u.Scheme != "" && u.Host != "" {
            // Is the argument a URL to blob storage?
            switch host := strings.ToLower(u.Host); true {
            // Azure Stack does not have the core.windows.net
            case strings.Contains(host, ".blob"):
                return common.ELocation.Blob()
            case strings.Contains(host, ".file"):
                return common.ELocation.File()
            case strings.Contains(host, ".dfs"):
                return common.ELocation.BlobFS()
            case strings.Contains(host, benchmarkSourceHost):
                return common.ELocation.Benchmark()
                // enable targeting an emulator/stack
            case IPv4Regex.MatchString(host):
                return common.ELocation.Unknown()//This is what gets returned in case of storage emulator URL.
            }

            if common.IsS3URL(*u) {
                return common.ELocation.S3()
            }
        }
    }

    return common.ELocation.Local()
}

回答2:

OK—after much futzing I was finally able to get this to work, using Azurite and PowerShell. It's clear that neither AzureCLI nor AzCopy are well-tested under emulation.

Here's a rough-and-tumble script that can be called from a pipeline:

[CmdletBinding()]
param(
  [Parameter(Mandatory)][string] $Container,
  [Parameter(Mandatory)][string] $Source
)

$Context = New-AzureStorageContext -Local
$BlobNames = Get-AzureStorageBlob -Context $Context -Container $Container | % { $_.Name }
$FilesToSync = gci $Source\* -Include RELEASES, Setup.exe
$Packages = gci $Source -Filter *.nupkg

$Packages | % {
  If (!($BlobNames.Contains($_.Name))) {
    $FilesToSync += $_
  }
}

$FilesToSync | Set-AzureStorageBlobContent -Context $Context -Container $Container -Force

Note that this is highly customized for my Squirrel deployments (*.nupkg, RELEASES, Setup.exe), so a person will want to adjust accordingly for his own environment.

Azurite can be set to always-on using a Scheduled Task to run this command every hour:

powershell -command "Start-Process azurite-blob.cmd -PassThru -ArgumentList '--blobHost 0.0.0.0'"

The argument sets Azurite to listen on any IP so that it can be reached from other computers on the network. I punched a hole in the firewall for ports 10000-10002.

Be careful to set the Task to run under the same account that was used to install Azurite, otherwise the Task won't be able to see azurite-blob.cmd (it's in %AppData%\npm, which is added to PATH during installation).

回答3:

The command line needs the --recursive option. See https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-blobs?toc=/azure/storage/blobs/toc.json

来源：https://stackoverflow.com/questions/62547525/azcopy-sync-command-is-failing

标签

azure

azure-storage

azcopy