Section 10.9.3 of the C# spec spells this out. The short version is that it's disallowed so that the user can be certain that conversions between reference types and interfaces succeed if and only if the reference type actually implements that interface, and that when that conversion takes place that the same object is actually being referenced.
Defining an implicit or explicit conversion between reference types gives the user the expectation that there will be a change in reference; after all, the same reference cannot be both types. On the other hand, the user does not have the same expectation for conversions between reference types and interface types.
User-defined conversions are not allowed to convert from or to interface-types. In particular, this restriction ensures that no user-defined transformations occur when converting to an interface-type, and that a conversion to an interface-type succeeds only if the object being converted actually implements the specified interface-type.