What DVCS support Unicode filenames?

Question

I\'m interested in trying out distributed version control systems. git sounds promising, but I saw a note somewhere for the Windows port of git that says \"don\'t use non-AS

Answer 1

Mercurial

On Linux, I think Mercurial just encodes in whatever is the system's encoding (correct me if I'm wrong). So best to set Linux up for UTF-8 for cross-platform compatibility. This is the default for many modern distributions.

On Windows, Mercurial (due to Python's byte string handling) uses the system code page. This just about guarantees bad cross-platform interoperation for non-ASCII characters.

fixutf8 Extension for Windows (prior to Mercurial 2.0)

There is an externally-created Mercurial extension called fixutf8 for Windows which properly handles all Unicode characters (even those outside the current code page) and encodes the filenames as UTF-8 in the Mercurial repository. It thus enables interoperation with Linux as long as Linux is using the UTF-8 encoding. I tried enabling it on my Windows set-up last week, and had a couple of problems with installation. Since then, one problem has been fixed. Now the only issue is that the binary Mercurial distributions are built with Python 2.4, while fixutf8 requires Mercurial to be built with Python 2.5 or higher to load fixutf8. I expect this will be resolved in the near future.

Mercurial 2.0 and later for Windows

fixutf8 seems to be incompatible with Mercurial 2.0 and later, according to the fixutf8 web page. See WindowsUTF8Plan for details on future solutions. I'm not sure when this is expected to be implemented.

Answer 2

According to this page: Bazaar, Codendi, CVSNT, Monotone, Perforce, Rational Team Concert, Subversion, Surround SCM, Synergy. But there are lots of 'Unknowns' on that page.

Answer 3

This is a really tricky problem. The problems come because either tools try to interpret filenames when they don't know the encoding for sure, or because they translate, but translate to a form which cannot handle all cases (e.g. ASCII or UTF-16). None of the main 3 OS's agree on how a filename is encoded either, making things even harder.

For a good understanding of the issues I suggest reading Mercurial's encoding strategy page. It describes how the various platforms vary, and why Mercurial has chosen the strategy it has.

If you really need to do this, then the most basic thing is that ALL systems need to be set-up to use UTF-8 filenames, and not one of the many Japanese code pages. This is easier said than done though, but once it is done, no system should need to translate the filenames into anything else.

No translation, no issues.

---

*: Yes, I know you can have a default system encoding, but this is not the same as a filesystem encoding. What happens when a filesystem is accessed by multiple systems or it is physically moved between systems?

Answer 4

git

August 2009:

The msysgit project is busy fixing UTF-8 support for Git on Windows. It might be fixed in the next release.

---

Update February 2012

UTF-8 is coming for msysgit, with commits like this one "Update less settings for UTF-8 "

From the Git for Windows Google+ page:

Karsten Blees' UTF-8 patches for Git for Windows has now been merged to 'devel'.
This means the upcoming release will support Unicode filenames!

---

Update April 2012

It's now released in mSysGit 1.7.10.

See the page Git for Windows Unicode Support.

Answer 5

Bazaar VCS works with unicode filenames internally. And it has very good support for unicode both on Linux and Windows.

Answer 6

See issue 80 in the same repository. In 2009, there was a discussion on the Git Mailing list (e.g. 1, 2) where the Git maintainer Junio Hamano asked some questions regarding this. I don't have it right here. By joining the thread in a constructive manner you might help in resolving the issue.

In the Java implementation JGit, we always use UTF-8 when we create textual metadata and filenames. That is the only way, but there are some things to consider.