Why do computers have byte-addressable memory, and not 4-byte-addressable memory (or 8-byte-addressable memory for 64bit)? Yeah, I see how it could be useful sometimes, it
Largely historical reasons - it has become the standard that CPUs understand. Here is a good discussion on it:
Generally, a size has to be chosen to be convenient for both data and machine instructions. 8 bits (256 values) is enough to accommodate common characters in English and some other languages. Designers of 8-bit processors presumably found that being able to encode 256 common instructions as one byte was a "reasonable tradeoff". And at the time, 8 bits was also generally enough to encode other things such as a pixel colour or screen coordinate. Having a byte size that is a power of 2 may also have been felt to be a "neater" design. It is interesting to note that, for example, Marxer, E. (1974), Elements of Data Processing, describes a byte as being either 6-bit and 8-bit depending on whether the computer was of the "octal" or "hexadecimal" type.
Certainly, other sizes were used in the early days.