I want to run my Java app and for a given workload be able to see:
Exactly this is where AOP helps. Aspects can be added/removed without changing code. If you are using Spring; create Aspect including JoinPoint, Advice listing the Classes and public methods that are to evaluate execution time. Add these beans to the Spring config. Else use AspectJ Container for non-spring application
There have been a couple profilers listed (The eclipse one and JProfiler). I just want to HIGHLY RECOMMEND that a profiler is one of the tools in your programming toolchest.
This is something most programmers pass over, but a profiler can solve entire classes of problems that are very difficult to solve otherwise.
I'm just saying (to everyone, not just the questioner) that if you haven't used a profiler go find one, download it and run it.
By the way, they are much more powerful than the static output of the java tools--although the java tools might be enough in this specific case. A profiler can tell you what each thread is doing and can make some pretty cool call graphics (flow diagram style) that will help you analyze code you didn't write.
Just find one, and use it for a week or two so that you know what it offers.
If you are willing to spend a little money,
JProfiler: http://www.ej-technologies.com/products/jprofiler/overview.html
is very good, it shows you % of time used, absolute time used, and # of invocations down to the method level. It understands EJB calls, web service calls, and will even show the SQL of jdbc calls. I use it frequently to find performance issues.
It has memory profiling too, but I find the cpu profiling much more useful.
Simplest approach for a program running in java 6 from Sun is to use the jvisualvm program in the jdk. Allows you to attach and profile without any special setup.
Java 1.7* comes bundled with Java Mission Control (jmc), which has 'Flight Recorder' feature which can be used to profile the method execution. Profiling results are displayed almost similar to AppDynamics - easy to spot the perf issues (esp. which methods are eating up all the CPU).
Though not detailed, but good blog post explaining the about Flight Recorder: http://hirt.se/blog/?p=364
* Not sure about the minor version
Take a look at Eclipse TPTP. They can provide exactly that and much more for any application started from Eclipse.