Edit: Comments at bottom. Also, this.
Here\'s what\'s kind of confusing me. My understanding is that if I have an enum like this...
Yes, special treatment. The JIT compiler is keenly aware of the way boxed value types work. Which is in general what makes value types acting a bit schizoid. Boxing involves creating a System.Object value that behaves exactly the same way as a value of a reference type. At that point, value type values no longer behave like values do at runtime. Which makes it possible, for example, to have a virtual method like ToString(). The boxed object has a method table pointer, just like reference types do.
The JIT compiler knows the method tables pointers for value types like int and bool up front. Boxing and unboxing for them is very efficient, it takes but a handful of machine code instructions. This needed to be efficient back in .NET 1.0 to make it competitive. A very important part of that is the restriction that a value type value can only be unboxed to the same type. This avoids the jitter from having to generate a massive switch statement that invokes the correct conversion code. All it has to do is to check the method table pointer in the object and verify that it is the expected type. And copy the value out of the object directly. Notable perhaps is that this restriction doesn't exist in VB.NET, its CType() operator does in fact generate code to a helper function that contains this big switch statement.
The problem with Enum types is that this cannot work. Enums can have a different GetUnderlyingType() type. In other words, the unboxed value has different sizes so simply copying the value out of the boxed object cannot work. Keenly aware, the jitter doesn't inline the unboxing code anymore, it generates a call to a helper function in the CLR.
That helper is named JIT_Unbox(), you can find its source code in the SSCLI20 source, clr/src/vm/jithelpers.cpp. You'll see it dealing with enum types specially. It is permissive, it allows unboxing from one enum type to another. But only if the underlying type is the same, you get an InvalidCastException if that's not the case.
Which is also the reason that Enum is declared as a class. Its logical behavior is of a reference type, derived enum types can be cast from one to another. With the above noted restriction on the underlying type compatibility. The values of an enum type have however very much the behavior of a value type value. They have copy semantics and boxing behavior.