Why do computers have byte-addressable memory, and not 4-byte-addressable memory (or 8-byte-addressable memory for 64bit)? Yeah, I see how it could be useful sometimes, it
Processors actually do access memory in quantities of 64-bit (x86 did since Pentium or so); 64-bit processors often have a 128-bit bus. Plus, in accessing main memory, you have bursts that fill an entire cache line, which is even larger units of memory.
It's only the addressing that is byte-based; this adds little overhead and is not excessive at all.
Today, you absolutely need byte-based addressing for networking protocols. Implementing TCP with word-based addressing would be difficult: what do you want read() to return if what you received where 17 bytes? Likewise, higher layers are byte-based: HTTP would be fairly difficult to implement if you get a request line like "GET / HTTP/1.0" be presented in units of four bytes. You essentially would have to split the words back into bytes with shift operations and such (which now the processors do in hardware, thanks to byte-based addressing).