I have come up against this problem a few times at inopportune moments:
As to why this still exists - MS doesn't consider it a priority, and values backwards compatibility over advancing their OS (at least in this instance).
A workaround I use is to use the "short names" for the directories in the path, instead of their standard, human-readable versions. So e.g. for C:\Program Files\
I would use C:\PROGRA~1\
You can find the short name equivalents using dir /x
.