Git clone: Redirect stderr to stdout but keep errors being written to stderr

Question

git clone writes its output to stderr as documented here. I can redirect this with the following command:

git clone https://myrepo          


        

Answer 1

A MingW update provide a new way to handle redirection with Git 2.15.x/2.16 (Q1 2018)

See commit b2f5571, commit 1a172e4, commit 3f94442 (01 Nov 2017) by Johannes Schindelin (dscho).
^{(Merged by Junio C Hamano -- gitster -- in commit 421f21c, 09 Nov 2017)}

mingw: add experimental feature to redirect standard handles

Particularly when calling Git from applications, such as Visual Studio's Team Explorer, it is important that stdin/stdout/stderr are closed properly.
However, when spawning processes on Windows, those handles must be marked as inheritable if we want to use them, but that flag is a global flag and may very well be used by other spawned processes which then do not know to close those handles.

Let's introduce a set of environment variables (GIT_REDIRECT_STDIN and friends) that specify paths to files, or even better, named pipes (which are similar to Unix sockets) and that are used by the spawned Git process.
This helps work around above-mentioned issue: those named pipes will be opened in a non-inheritable way upon startup, and no handles are passed around (and therefore no inherited handles need to be closed by any spawned child).

This feature shipped with Git for Windows (marked as experimental) since v2.11.0(2), so it has seen some serious testing in the meantime.

The Git documentation now includes:

GIT_REDIRECT_STDIN:
GIT_REDIRECT_STDOUT:
GIT_REDIRECT_STDERR:

Windows-only: allow redirecting the standard input/output/error handles to paths specified by the environment variables. This is particularly useful in multi-threaded applications where the canonical way to pass standard handles via CreateProcess() is not an option because it would require the handles to be marked inheritable (and consequently every spawned process would inherit them, possibly blocking regular Git operations).

The primary intended use case is to use named pipes for communication (e.g. \\.\pipe\my-git-stdin-123).

And it adds:

mingw: optionally redirect stderr/stdout via the same handle

The "2>&1" notation in Powershell and in Unix shells implies that stderr is redirected to the same handle into which stdout is already written.

Let's use this special value to allow the same trick with GIT_REDIRECT_STDERR and GIT_REDIRECT_STDOUT: if the former's value is 2>&1, then stderr will simply be written to the same handle as stdout.

The functionality was suggested by Jeff Hostetler.

Example of usage:$env:GIT_REDIRECT_STDERR = '2>&1'

Answer 2

Generally speaking:

• Console (terminal) applications - whether on Windows or on Unix-like platforms - only have two output streams at their disposal:

  • stdout (standard output) - this is where data ("return values") goes.
  • stderr (standard error) - this is where error messages and - for lack of additional streams - anythings else that isn't data goes, such as progress and status information.

• Therefore, you cannot and shouldn't infer success vs. failure from the presence of stderr output.

  • Instead, you must rely solely on an application's process exit code:

    • 0 indicates success
    • any nonzero value indicates failure

  • In PowerShell, the **process exit code is reflected in automatic variable $LASTEXITCODE**.

---

Specifically, this means:

• Given git's stderr output lines, you cannot infer whether they represent error message or other kind of non-data information, such as progress or status messages, which git frequently uses.

  • Telling git to categorically redirect its stderr output to stdout (by setting environment variable GIT_REDIRECT_STDERR to string 2>&1; $env:GIT_REDIRECT_STDERR = '2>&1' in PowerShell) does not help, because error messages and progress/status messages alike are then sent there.

• As stated, you should only infer failure from a nonzero exit code.

---

A pragmatic approach is to do the following:

# Invoke git and capture both its stdout and stderr streams.
$result = git clone https://myrepo c:\repo 2>&1
# Throw an error, if git indicated failure.
if ($LASTEXITCODE) { Throw "git failed (exit code: $LASTEXITCODE):`n$($result -join "`n")" }                                            #`
# Output the captured result, as an array of strings
$result | % ToString 

Note:

• | % ToString ensures that only strings are output, given that the stderr lines (via stream 2) redirected to (>) the success output stream (&1) are wrapped in System.Management.Automation.ErrorRecord instances.

• 2>&1 can have unexpected side effects - see this answer for background information.

• Better integration of external-program calls into PowerShell's error handling is the subject of this RFC draft, notably to have the option to abort execution automatically on encountering a nonzero exit code.

Answer 3

You can get rid from the stderr.

by this command:

git clone https://myrepo c:\repo 2>$null

By doing that all stderr will not show.

You can not display the progress and throw away only the errors . If the command failed all output is stderr if succeed stdout

Edit: Looks like git command output will be always stderr even when the command succeed only on Windows. T.

Answer 4

I have modified the Invoke-Git to suit my needs better.

From many of the posts that I have read while looking for a solution I'm guessing more than a few people can use this.

Enjoy.

This version that will:

• Execute the Git command passed in (assumes Git is in the execution path already.)
• If all goes well then all output (stdout and stderr) is displayed on the host, NOT via stderr.
• Check the $LASTEXITCODE to see if there was actually an error. If there was an error then all the output is thrown to the caller to deal with.

<#
.Synopsis
    Invoke git, handling its quirky stderr that isn't error

.Outputs
    Git messages

.Example
    Invoke-Git push

.Example
    Invoke-Git "add ."
#>
function Invoke-Git
{
param(
[Parameter(Mandatory)]
[string] $Command )

    try {
        # I could do this in the main script just once, but then the caller would have to know to do that 
        # in every script where they use this function.
        $old_env = $env:GIT_REDIRECT_STDERR
        $env:GIT_REDIRECT_STDERR = '2>&1'

        Write-Host -ForegroundColor Green "`nExecuting: git $Command "
        $output = Invoke-Expression "git $Command "
        if ( $LASTEXITCODE -gt 0 )
        {
            # note: No catch below (only the try/finally). Let the caller handle the exception.
            Throw "Error Encountered executing: 'git $Command '"
        }
        else
        {
            # because $output probably has miultiple lines (array of strings), by piping it to write-host we get multiple lines.
            $output | Write-Host 
        }
    }
    # note: No catch here. Let the caller handle it.
    finally
    {
        $env:GIT_REDIRECT_STDERR = $old_env
    }
}

Answer 5

I use this script to run git commands. Since git will write to stderr even if successful (e.g. pull when in sync), this handles those cases and writes out first line of output, which is usually what you need to know.

<#
.Synopsis
    Invoke git, handling its quirky stderr that isn't error

.Outputs
    Git messages, and lastly the exit code

.Example
    Invoke-Git push

.Example
    Invoke-Git "add ."
#>
function Invoke-Git
{
param(
[Parameter(Mandatory)]
[string] $Command )

    try {

        $exit = 0
        $path = [System.IO.Path]::GetTempFileName()

        Invoke-Expression "git $Command 2> $path"
        $exit = $LASTEXITCODE
        if ( $exit -gt 0 )
        {
            Write-Error (Get-Content $path).ToString()
        }
        else
        {
            Get-Content $path | Select-Object -First 1
        }
        $exit
    }
    catch
    {
        Write-Host "Error: $_`n$($_.ScriptStackTrace)"
    }
    finally
    {
        if ( Test-Path $path )
        {
            Remove-Item $path
        }
    }
}

Answer 6

Here's yet another take which might serve as inspiration for some. As others pointed out redirecting and checking exit code works well. Things which are different from other answers:

• Redirecting to a file is a bit annoying because as long as git is running you just don't see anything happening, so just return the output and use a separate error message. It's a tradeoff, but I like it more.
• Use Write-Verbose instead of Write-Host since the former is configurable
• Use Write-Error to produce an error instead of the harsh Throw. Makes things like ErrorAction, ErrorVariable work as expected.
• Support setting the directory in which to run, like you'd use -C instead of having to cd into directories, but also support that for commands which normally do not have that functionality. So you can do igit -dir some/path stash. I use this mainly in automated scripts where it's otherwise annoying to have to cd into directories.
• Use ValueFromRemainingArguments etc so commands can be passed as if you're writing a git command directly so not requiring a string, but still allowing it. So igit checkout master works like igit 'checkout master'. Almost, that is, because standard PS caveats apply: quoting so you still need an actual string if quotes need to be passed to the underlying command, i.e. igit log '--format="%h %d"'. And PS doesn't require you to type full parameter names meaning igit push -vwill be interpreted asigit push -Verboseinstead of passing-vi.e. verbose push to git. Use double dash to deal with thatigit -- push -v` or write quotes after all.

Code:

<#
.SYNOPSIS
Run git, Powershell-style.
.DESCRIPTION
By default some git commands (clone, checkout, ...) write a part of their
output to stderr, resulting in PS treating that as an error.
Here we work around that by redirecting stderr and using git's exit code
to check if something was actually wrong, and use Write-Error if that's the case,
i.e. standard PS error handling which works with -ErrorAction/-ErrorVariable etc.
The command can be passed as a string or as separate strings.
Additionally takes a $Directory argument which when used has the same effect as git -C,
but also works for clone/stash/submodule/... commands making it easier to automate those.
The $Git argument can be used to specify the executable.
.EXAMPLE
Invoke-Git status
Invoke-Git -Directory some/path status
Invoke-Git 'push -v'
Invoke-Git -Verbose -- push -v  # Pass that last -v to git.
#>
function Invoke-Git {
  [CmdletBinding()]
  param(
    [Parameter()] [Alias('Dir')] [String] $Directory = $null,
    [Parameter()] [String] $Git = 'git',
    [Parameter(Mandatory, Position=0, ValueFromRemainingArguments=$true)] [string] $Command
  )
  try {
    $commandParts = $Command.Split(' ')
    $subCommand = $commandParts[0]
    if ($Directory -and $subCommand -eq 'clone') {
      # To make all commands look alike handle this one as well.
      $Command = ($commandParts + @($Directory)) -join ' '
    } elseif ($Directory -and @('submodule', 'stash', 'init') -eq $subCommand) {
      # These currently require one to be in the git directory so go there.
      $currentDir = Get-Location
      cd $Directory
    } elseif ($Directory) {
      if ($commandParts -eq '-C') {
        # Not an error, git will pick the last one, but unexpected.
        Write-Warning 'Better use either -Directory or -C, not both'
      }
      $Command = "-C $Directory " + $Command
    }
    Write-Verbose "Invoke-Git on '$Directory' with command '$Command'"
    $gitRedirection = $env:GIT_REDIRECT_STDERR
    $env:GIT_REDIRECT_STDERR = '2>&1'
    # Deliberately not getting output here: while this means we cannot pass the actual error to Write-Error,
    # it does result in all commands being shown 'live'. Otherwise when doing a clone for instance,
    # nothing gets displayed while git is doing it's thing which is unexepected and too different from normal usage.
    Invoke-Expression "$Git $Command"
    if ($LASTEXITCODE -ne 0) {
      Write-Error "git exited with code $LASTEXITCODE"
    }
  } finally {
    $env:GIT_REDIRECT_STDERR = $gitRedirection
    if ($currentDir) {
      cd $currentDir
    }
  }
}

New-Alias -Name IGit -Value Invoke-Git -ErrorAction SilentlyContinue