Custom RoboCopy Progress Bar in PowerShell

Question

I\'m interested in a PowerShell script that copies a large amount of files from a server daily and I\'m interested in implementing a in-console progress bar like

Answer 1

Here is native PowerShell GUI version of RoboCopy. (NO EXE file)

I hope it helps some one.

https://gallery.technet.microsoft.com/PowerShell-Robocopy-GUI-08c9cacb

FYI : Are there any one who can combine PowerCopy GUI tool with Copy-WithProgress bar?

Answer 2

This is the code-snippet I finally used for such task:

$fileName = 'test.txt'
$fromDir  = 'c:\'
$toDir    = 'd:\'

$title = $null
&robocopy "$fromDir" "$toDir" "$fileName" /z /mt /move /w:3 /r:10 /xo | %{
    $data = $_.Split("`t")
    if ($title -and $data[0] -match '\d+(?=%)') {
        Write-Progress $title -Status $data -PercentComplete $matches[0]
    }
    if($data[4]) {$title = $data[4]}
}
Write-Progress $title -complete

Answer 3

Progress bars are nice and all but when copying hundreds of files, showing progress slows down the operation, in some cases quite a bit. It's one reason that the robocopy help says for the /MT flag to redirect output to log for better performance.

Answer 4

These solutions are great but a quick and easy way to get a floating progress for all the files easily is as follows:

robocopy <source> <destination> /MIR /NDL /NJH /NJS | %{$data = $_.Split([char]9); if("$($data[4])" -ne "") { $file = "$($data[4])"} ;Write-Progress "Percentage $($data[0])" -Activity "Robocopy" -CurrentOperation "$($file)"  -ErrorAction SilentlyContinue; }

Answer 5

I wrote a PowerShell function called Copy-WithProgress that will achieve what you are after. Since you specifically stated that you were using robocopy, I built a PowerShell function that encapsulates the robocopy functionality (at least, parts of it).

Allow me to show you how it works. I've also recorded and posted a YouTube video demonstrating how the function is designed to work, and invoking a test run.

The function is divided into regions:

• Common robocopy parameters
• Staging (where the robocopy job size is calculated)
• Copy (where the robocopy job is kicked off)
• Progress bar (where the robocopy progress is monitored)
• Function output (where some useful statistics are outputted, for use in the rest of your script)

There are several parameters on the function.

• Source: The source directory
• Destination: The destination directory
• Gap: The "inter-packet gap" in milliseconds supported by robocopy, which artificially slows down the copy, for testing)
• ReportGap: The interval (in milliseconds) to check on robocopy progress

At the bottom of the script (after the function definition), is a complete example of how to call it. It should work on your computer, since everything is variable-ized. There are five steps:

1. Generate a random source directory
2. Generate a destination directory
3. Call the Copy-WithProgress function
4. Create some additional source files (to emulate changes over time)
5. Call the Copy-WithProgress function again, and validate only changes are replicated

Here is a screenshot of what the function's output looks like. You can leave off the -Verbose parameter, if you do not want all of the debugging information. A PSCustomObject is returned, by the function, which tells you:

1. How many bytes were copied
2. How many files were copied

Here is a screenshot of the PowerShell Progress Bar in the PowerShell ISE, and the PowerShell Console Host.

Here is the code:

function Copy-WithProgress {
    [CmdletBinding()]
    param (
            [Parameter(Mandatory = $true)]
            [string] $Source
        , [Parameter(Mandatory = $true)]
            [string] $Destination
        , [int] $Gap = 200
        , [int] $ReportGap = 2000
    )
    # Define regular expression that will gather number of bytes copied
    $RegexBytes = '(?<=\s+)\d+(?=\s+)';

    #region Robocopy params
    # MIR = Mirror mode
    # NP  = Don't show progress percentage in log
    # NC  = Don't log file classes (existing, new file, etc.)
    # BYTES = Show file sizes in bytes
    # NJH = Do not display robocopy job header (JH)
    # NJS = Do not display robocopy job summary (JS)
    # TEE = Display log in stdout AND in target log file
    $CommonRobocopyParams = '/MIR /NP /NDL /NC /BYTES /NJH /NJS';
    #endregion Robocopy params

    #region Robocopy Staging
    Write-Verbose -Message 'Analyzing robocopy job ...';
    $StagingLogPath = '{0}\temp\{1} robocopy staging.log' -f $env:windir, (Get-Date -Format 'yyyy-MM-dd HH-mm-ss');

    $StagingArgumentList = '"{0}" "{1}" /LOG:"{2}" /L {3}' -f $Source, $Destination, $StagingLogPath, $CommonRobocopyParams;
    Write-Verbose -Message ('Staging arguments: {0}' -f $StagingArgumentList);
    Start-Process -Wait -FilePath robocopy.exe -ArgumentList $StagingArgumentList -NoNewWindow;
    # Get the total number of files that will be copied
    $StagingContent = Get-Content -Path $StagingLogPath;
    $TotalFileCount = $StagingContent.Count - 1;

    # Get the total number of bytes to be copied
    [RegEx]::Matches(($StagingContent -join "`n"), $RegexBytes) | % { $BytesTotal = 0; } { $BytesTotal += $_.Value; };
    Write-Verbose -Message ('Total bytes to be copied: {0}' -f $BytesTotal);
    #endregion Robocopy Staging

    #region Start Robocopy
    # Begin the robocopy process
    $RobocopyLogPath = '{0}\temp\{1} robocopy.log' -f $env:windir, (Get-Date -Format 'yyyy-MM-dd HH-mm-ss');
    $ArgumentList = '"{0}" "{1}" /LOG:"{2}" /ipg:{3} {4}' -f $Source, $Destination, $RobocopyLogPath, $Gap, $CommonRobocopyParams;
    Write-Verbose -Message ('Beginning the robocopy process with arguments: {0}' -f $ArgumentList);
    $Robocopy = Start-Process -FilePath robocopy.exe -ArgumentList $ArgumentList -Verbose -PassThru -NoNewWindow;
    Start-Sleep -Milliseconds 100;
    #endregion Start Robocopy

    #region Progress bar loop
    while (!$Robocopy.HasExited) {
        Start-Sleep -Milliseconds $ReportGap;
        $BytesCopied = 0;
        $LogContent = Get-Content -Path $RobocopyLogPath;
        $BytesCopied = [Regex]::Matches($LogContent, $RegexBytes) | ForEach-Object -Process { $BytesCopied += $_.Value; } -End { $BytesCopied; };
        $CopiedFileCount = $LogContent.Count - 1;
        Write-Verbose -Message ('Bytes copied: {0}' -f $BytesCopied);
        Write-Verbose -Message ('Files copied: {0}' -f $LogContent.Count);
        $Percentage = 0;
        if ($BytesCopied -gt 0) {
           $Percentage = (($BytesCopied/$BytesTotal)*100)
        }
        Write-Progress -Activity Robocopy -Status ("Copied {0} of {1} files; Copied {2} of {3} bytes" -f $CopiedFileCount, $TotalFileCount, $BytesCopied, $BytesTotal) -PercentComplete $Percentage
    }
    #endregion Progress loop

    #region Function output
    [PSCustomObject]@{
        BytesCopied = $BytesCopied;
        FilesCopied = $CopiedFileCount;
    };
    #endregion Function output
}

# 1. TESTING: Generate a random, unique source directory, with some test files in it
$TestSource = '{0}\{1}' -f $env:temp, [Guid]::NewGuid().ToString();
$null = mkdir -Path $TestSource;
# 1a. TESTING: Create some test source files
1..20 | % -Process { Set-Content -Path $TestSource\$_.txt -Value ('A'*(Get-Random -Minimum 10 -Maximum 2100)); };

# 2. TESTING: Create a random, unique target directory
$TestTarget = '{0}\{1}' -f $env:temp, [Guid]::NewGuid().ToString();
$null = mkdir -Path $TestTarget;

# 3. Call the Copy-WithProgress function
Copy-WithProgress -Source $TestSource -Destination $TestTarget -Verbose;

# 4. Add some new files to the source directory
21..40 | % -Process { Set-Content -Path $TestSource\$_.txt -Value ('A'*(Get-Random -Minimum 950 -Maximum 1400)); };

# 5. Call the Copy-WithProgress function (again)
Copy-WithProgress -Source $TestSource -Destination $TestTarget -Verbose;

Answer 6

I ended up using this based on Amrinder's suggested answer:

robocopy.exe $Source $Destination $PatternArg $MirrorArg /NDL /NJH /NJS | ForEach-Object -Process {
    $data = $_.Split([char]9);
    if (($data.Count -gt 4) -and ("$($data[4])" -ne ""))
    {
        $file = "$($data[4])"
        Write-Progress "Percentage $($data[0])" -Activity "Robocopy" -CurrentOperation "$($file)" -ErrorAction SilentlyContinue; 
    }
    else
    {
        Write-Progress "Percentage $($data[0])" -Activity "Robocopy" -CurrentOperation "$($file)"
    }
}
# Robocopy has a bitmask set of exit codes, so only complain about failures:
[int] $exitCode = $global:LastExitCode;
[int] $someCopyErrors = $exitCode -band 8;
[int] $seriousError = $exitCode -band 16;
if (($someCopyErrors -ne 0) -or ($seriousError -ne 0))
{
    Write-Error "ERROR: robocopy failed with a non-successful exit code: $exitCode"
    exit 1
}

Fyi, Bill