UTF-8 output from PowerShell

Question

I\'m trying to use Process.Start with redirected I/O to call PowerShell.exe with a string, and to get the output back, all in UTF-8. But I don\'t s

Answer 1

This is a bug in .NET. When PowerShell launches, it caches the output handle (Console.Out). The Encoding property of that text writer does not pick up the value StandardOutputEncoding property.

When you change it from within PowerShell, the Encoding property of the cached output writer returns the cached value, so the output is still encoded with the default encoding.

As a workaround, I would suggest not changing the encoding. It will be returned to you as a Unicode string, at which point you can manage the encoding yourself.

Caching example:

102 [C:\Users\leeholm]
>> $r1 = [Console]::Out

103 [C:\Users\leeholm]
>> $r1

Encoding                                          FormatProvider
--------                                          --------------
System.Text.SBCSCodePageEncoding                  en-US



104 [C:\Users\leeholm]
>> [Console]::OutputEncoding = [System.Text.Encoding]::UTF8

105 [C:\Users\leeholm]
>> $r1

Encoding                                          FormatProvider
--------                                          --------------
System.Text.SBCSCodePageEncoding                  en-US

Answer 2

Not an expert on encoding, but after reading these...

• http://blogs.msdn.com/b/powershell/archive/2006/12/11/outputencoding-to-the-rescue.aspx
• http://technet.microsoft.com/en-us/library/hh847796.aspx
• http://www.johndcook.com/blog/2008/08/25/powershell-output-redirection-unicode-or-ascii/

... it seems fairly clear that the $OutputEncoding variable only affects data piped to native applications.

If sending to a file from withing PowerShell, the encoding can be controlled by the -encoding parameter on the out-file cmdlet e.g.

write-output "hello" | out-file "enctest.txt" -encoding utf8

Nothing else you can do on the PowerShell front then, but the following post may well help you:.

• http://blogs.msdn.com/b/ddietric/archive/2010/11/08/decoding-standard-output-and-standard-error-when-redirecting-to-a-gui-application.aspx

Answer 3

Set the [Console]::OuputEncoding as encoding whatever you want, and print out with [Console]::WriteLine.

If powershell ouput method has a problem, then don't use it. It feels bit bad, but works like a charm :)

Answer 4

Spent some time working on a solution to my issue and thought it may be of interest. I ran into a problem trying to automate code generation using PowerShell 3.0 on Windows 8. The target IDE was the Keil compiler using MDK-ARM Essential Toolchain 5.24.1. A bit different from OP, as I am using PowerShell natively during the pre-build step. When I tried to #include the generated file, I received the error

fatal error: UTF-16 (LE) byte order mark detected '..\GITVersion.h' but encoding is not supported

I solved the problem by changing the line that generated the output file from:

out-file -FilePath GITVersion.h -InputObject $result

to:

out-file -FilePath GITVersion.h -Encoding ascii -InputObject $result