fstream::open() Unicode or Non-Ascii characters don't work (with std::ios::out) on Windows

Question

In a C++ project, I want to open a file (fstream::open()) (which seems to be a major problem). The Windows build of my program fails miserably.

Answer 1

In Microsoft implementations of STL, there's a non-standard extension (overload) to allow unicode support for UTF-16 encoded strings.

Just pass UTF-16 encoded std::wstring to fstream::open(). This is the only way to make it work with fstream.

You can read more on what I find to be the easiest way to support unicode on windows here: http://utf8everywhere.org/

Answer 2

Using the standard APIs (such as std::fstream) on Windows you can only open a file if the filename can be encoded using the currently set "ANSI Codepage" (CP_ACP).

This means that there can be files which simply cannot be opened using these APIs on Windows. Unless Microsoft implements support for setting CP_ACP to CP_UTF8 then this cannot be using Microsoft's CRT or C++ standard library implementation.

(Windows has had a feature called "short" filenames where, when enabled, every file on the drive had an ASCII filename that can be used via standard APIs. However this feature is going away so it does not represent a viable solution.)