We are using JMeter in a heterogeneous environment and we would like to measure CPU and memory while we are producing heavy load on a server where our service or web application
You can use our JMeter perfmon plugin to measure CPU, memory and a lot of other metrics:
From the java perspective, you can use jconsole or write your own code to monitor the memory usage.
One you install the Perfmon plugin, I also had to install the server agent, and open the firewall port 4444.
Here's a link which I found super useful.
http://grosan.co.uk/web-application-performance-testing-using-apache-jmeter-part-2-monitor-server-resources/
Perhaps you can take a look to collectd if you run in some unix/linux flavour. It will bring you not only CPU but also memory, Disk IO, network and a good number of plugins available.
You could take a look at Cacti.
It could collect resource usages like CPU, MEM easily with built-in templates and display them in nice graphics. Both Windows and Uni* are supported.