michael.rp almost had it, but no quotes on the environment string for Windows. The same solution for Linux appears in the link supplied by jerry. For Windows:
set RDOCOPT=--encoding=UTF-8
I put this in the RubyInstaller\Ruby1.9.3\setup_environment.bat file so it is always there.