I think John Carmack said it best with:
The biggest problem is that Java is really slow. On a pure cpu / memory / display / communications level, most modern cell phones should be considerably better gaming platforms than a Game Boy Advanced. With Java, on most phones you are left with about the CPU power of an original 4.77 mhz IBM PC, and lousy control over everything.
[...snip...]
Write-once-run-anywhere. Ha. Hahahahaha. We are only testing on four platforms right now, and not a single pair has the exact same quirks. All the commercial games are tweaked and compiled individually for each (often 100+) platform. Portability is not a justification for the awful performance.
(source)
Granted, he was talking about mobile platforms, but I've found similar problems with Java as a whole coming from a C++ background.
I miss being able to allocate memory on the Stack/Heap on my own terms.