Can I safely attempt to create the same directory from two different threads, without having one of them throw an exception, or run into other issues?
From the MSDN docs on Directory:
Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.
Therefore, as CreateDirectory is static, yes, it is thread safe.
That said: as @JaredPar points out, thread safety issues are not the only reason that a method can throw exceptions. There are a multitude of reasons why a filesystem call might throw an exception (under any circumstances, multithreaded or not), and you need to account for those.
By saying it is thread safe I (and MSDN) only imply the very literal interpretation of that, meaning "this method does not modify shared program state in a way that could cause invalid state, race conditions, or other adverse effects commonly associated with unsafe multithreaded code"