Core dump taken with gcore, jmap conversion to hprof file format fails with Error message

Question

We recently had one of our JVM\'s crash, leaving behind a core dump file produced by the gcore command. We want to have a look at the contents of the file and find out exact

Answer 1

This was bothering the heck out of me as I had a core file that represented a heap that I needed to analyze, but I was constantly seeing the exception message below:

sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names (libjvm.so, libjvm_g.so, gamma_g)

Copying the jre from my source machine (the machine where the core file was obtained) on to the exact same folder in the destination machine, and then running jmap with that java location as an argument worked for me.

So here are the steps to try in case someone else runs into this:
1. Connect to the core file through gdb and confirm the location of java binary which the running process was using:

    gdb --core=</path/to/core-file>

2. The above output will end with something like

[New Thread 22748]
**Core was generated by `/opt/blah/location/jre/bin/java -Xmx...'.**

3. Make sure you copy the matching version of the jre into the /opt/blah/location/ directory

4. Then launch jmap as:

   /opt/jdk1.8.0_09/bin/jmap -heap /opt/blah/location/jre/bin/java /path/to/core-file
   

   This should connect to the core file successfully and print out heap statistics. If it does, then you have successfully read the core file

5. From that point on, you can generate the hprof from the core file successfully using:

   /opt/jdk1.8.0_09/bin/jmap -dump:format=b,file=my-file.hprof /opt/blah/location/jre/bin/java /path/to/core-file
   

Answer 2

By the way, jvisualvm can load core dumps directly. But you must use the same jvm that created the core file.

Answer 3

Was the core file larger than 2GB? If so, you could be having an issue with the Linux build of libsaproc.so that comes with the JVM.

Run your command again, but like this:

strace -o out.txt -f $yourOriginalCommand

Then 'grep core.2878 out.txt' and look for an error on the open() syscall. Did it return an error (E_XXXXX) or a file handle number?