The other answers given within this thread, as well as the discussions given here are instructive:
(1) Why is a cast required for byte subtraction in C#?
(2) byte + byte = int… why?
(3) Why is a cast required for byte subtraction in C#?
But just to throw another wrinkle into it, it can depend on which operators you use. The increment (++) and decrement (--) operators as well as the addition assignment (+=) and subtraction assignment (-=) operators are overloaded for a variety of numeric types, and they perform the extra step of converting the result back to the operand's type when returning the result.
For example, using short:
short s = 0;
s++; // <-- Ok
s += 1; // <-- Ok
s = s + 1; // <-- Compile time error!
s = s + s; // <-- Compile time error!
Using byte:
byte b = 0;
b++; // <-- Ok
b += 1; // <-- Ok
b = b + 1; // <-- Compile time error!
b = b + b; // <-- Compile time error!
If they didn't do it this way, calls using the increment operator (++) would be impossible and calls to the addition assignment operator would be awkward at best, e.g.:
short s
s += (short)1;
Anyway, just another aspect to this whole discussion...