问题
My application failed for a customer crashing the JVM
I have these details:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb6b48b12, pid=31447, tid=31508
#
# JRE version: OpenJDK Runtime Environment (9.0+11) (build 9.0.1+11)
# Java VM: OpenJDK Client VM (9.0.1+11, mixed mode, serial gc, linux-)
# Problematic frame:
# V [libjvm.so+0x1d9b12] AdaptiveWeightedAverage::sample(float)+0x55
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /mnt/system/config/Apps/SongKong/songkong/hs_err_pid31447.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Does it look familar to anyone, I have search both https://bugs.java.com/bugdatabase/ and https://bugs.openjdk.java.net/secure/Dashboard.jspa without finding anything.
Here is the start of the hs_err_id log file
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb6aacb12, pid=13032, tid=13074
#
# JRE version: OpenJDK Runtime Environment (9.0+11) (build 9.0.1+11)
# Java VM: OpenJDK Client VM (9.0.1+11, mixed mode, serial gc, linux-)
# Problematic frame:
# V [libjvm.so+0x1d9b12] AdaptiveWeightedAverage::sample(float)+0x55
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
--------------- S U M M A R Y ------------
Command Line: -Duser.home=/mnt/disk1/share -Xms150m -Xmx900m -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog -Dorg.jboss.logging.provider=jdk -Djava.util.logging.config.class=com.jthink.songkong.logging.StandardLogging --add-opens=java.base/java.lang=ALL-UNNAMED lib/songkong-6.8.jar -r
Host: Marvell PJ4Bv7 Processor rev 1 (v7l), 1 cores, 1G, Buildroot 2014.11-rc1
Time: Fri Feb 7 01:50:35 2020 GMT elapsed time: 53621 seconds (0d 14h 53m 41s)
--------------- T H R E A D ---------------
Current thread (0xb6657d00): VMThread "VM Thread" [stack: 0x7b7e9000,0x7b869000] [id=13074]
Stack: [0x7b7e9000,0x7b869000], sp=0x7b8677a0, free space=505k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x1d9b12] AdaptiveWeightedAverage::sample(float)+0x55
[error occurred during error reporting (printing native stack), id 0xb]
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000a
Register to memory mapping:
r0 = 0x00000000
0x00000000 is an unknown value
r1 = 0x00000100
0x00000100 is an unknown value
r2 = 0x000001fc
0x000001fc is an unknown value
r3 = 0x00000023
0x00000023 is an unknown value
r4 = 0x783bf970
0x783bf970 is an unknown value
r5 = 0x00000000
0x00000000 is an unknown value
r6 = 0x0e6a0000
0x0e6a0000 is an unknown value
r7 = 0x00000000
0x00000000 is an unknown value
r8 = 0x783bf940
0x783bf940 is an unknown value
r9 = 0x0e6a0000
0x0e6a0000 is an unknown value
r10 = 0x7b8679ec
0x7b8679ec is an unknown value
fp = 0xb6657c38
0xb6657c38 is an unknown value
r12 = 0xfffffefe
0xfffffefe is an unknown value
sp = 0x7b8677a0
0x7b8677a0 is an unknown value
lr = 0xb6c62daf
0xb6c62daf: <offset 0x0038fdaf> in /mnt/system/config/Apps/SongKong/songkong/jre/lib/client/libjvm.so at 0xb68d3000
pc = 0xb6aacb12
0xb6aacb12: <offset 0x001d9b12> in /mnt/system/config/Apps/SongKong/songkong/jre/lib/client/libjvm.so at 0xb68d3000
Registers:
r0 = 0x00000000
r1 = 0x00000100
r2 = 0x000001fc
r3 = 0x00000023
r4 = 0x783bf970
r5 = 0x00000000
r6 = 0x0e6a0000
r7 = 0x00000000
r8 = 0x783bf940
r9 = 0x0e6a0000
r10 = 0x7b8679ec
fp = 0xb6657c38
r12 = 0xfffffefe
sp = 0x7b8677a0
lr = 0xb6c62daf
pc = 0xb6aacb12
cpsr = 0x80000030
Top of Stack: (sp=0x7b8677a0)
0x7b8677a0: 00000000 404e0000 00000001 000b6448
0x7b8677b0: 7b8677b8 b6c62daf b6d4c8b8 783bf910
0x7b8677c0: 00000000 b660d800 7b867808 b6d1f000
0x7b8677d0: b662f310 b6c62f43 7b8679f6 b6cac9cc
0x7b8677e0: 7b8677f0 b6d1f000 7b8679ec b6b1d987
0x7b8677f0: 7b867820 7b8679f0 7b867800 b6a068a3
0x7b867800: 00000400 7b867820 00000000 00000012
0x7b867810: b660d800 00000000 00000012 b660d800
Instructions: (pc=0xb6aacb12)
0xb6aacaf2: ee064603 eeb83a10 ee377a46 ee286ac7
0xb6aacb02: ee667a07 ee876a86 eec67a27 ee777aa7
0xb6aacb12: edc47a87 46bd7a00 8b02ecbd 2064bd98
0xb6aacb22: f9e1f1fd 6a00edd4 0000e7db 480642c8
VM_Operation (0x77c5daa0): GenCollectForAllocation, mode: safepoint, requested by thread 0x77a0d790
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x742a7448 JavaThread "Thread-513" daemon [_thread_blocked, id=11706, stack(0x76a70000,0x76ac0000)]
0x73db7458 JavaThread "Thread-510" daemon [_thread_blocked, id=11027, stack(0x773b6000,0x77406000)]
0x732b66f0 JavaThread "Thread-509" daemon [_thread_blocked, id=10757, stack(0x77b70000,0x77bc0000)]
0x762428e0 JavaThread "Thread-508" daemon [_thread_blocked, id=10130, stack(0x79310000,0x79360000)]
0x74284f48 JavaThread "Thread-503" daemon [_thread_blocked, id=8094, stack(0x78210000,0x78260000)]
0x76d60e10 JavaThread "qtp9318356-11036" [_thread_blocked, id=4101, stack(0x78410000,0x78460000)]
0x762dc8f0 JavaThread "qtp9318356-11035" [_thread_blocked, id=4100, stack(0x79bc0000,0x79c10000)]
0x783bf910 JavaThread "qtp9318356-9925" [_thread_blocked, id=22685, stack(0x78260000,0x782b0000)]
0x732170a8 JavaThread "qtp9318356-5567" [_thread_blocked, id=9426, stack(0x76810000,0x76860000)]
0x760e2708 JavaThread "qtp9318356-5564" [_thread_blocked, id=9423, stack(0x768b0000,0x76900000)]
0x77a0d790 JavaThread "Worker:12" [_thread_blocked, id=18002, stack(0x77c10000,0x77c60000)]
0x77a0daf0 JavaThread "pool-1639-thread-1" [_thread_blocked, id=17984, stack(0x77186000,0x771d6000)]
0x76db5e88 JavaThread "Worker:11" [_thread_blocked, id=17983, stack(0x77316000,0x77366000)]
0x783c9df8 JavaThread "Worker:10" [_thread_blocked, id=17982, stack(0x77406000,0x77456000)]
0x77a25388 JavaThread "SongLoaderWorker:9" [_thread_blocked, id=17942, stack(0x77366000,0x773b6000)]
0x76d6bce8 JavaThread "com.jthink.songkong.server.callback.ServerFixSongs:8" [_thread_blocked, id=17922, stack(0x793b0000,0x79400000)]
0x769c7cb8 JavaThread "pool-1636-thread-2" [_thread_blocked, id=17764, stack(0x77276000,0x772c6000)]
0x732c50d8 JavaThread "pool-1636-thread-1" [_thread_blocked, id=17763, stack(0x77036000,0x77086000)]
0x7690e140 JavaThread "pool-1601-thread-2" [_thread_blocked, id=11810, stack(0x771d6000,0x77226000)]
0x769077c8 JavaThread "pool-1601-thread-1" [_thread_blocked, id=11809, stack(0x77226000,0x77276000)]
0x769368a8 JavaThread "MappedByteBufferSource Async Flush Thread" daemon [_thread_blocked, id=8920, stack(0x76fe6000,0x77036000)]
0x77d9d990 JavaThread "com.jthink.songkong.fileloader.LoadFolderWorker:3" [_thread_blocked, id=380, stack(0x77eb0000,0x77f00000)]
0x77d940d8 JavaThread "pool-429-thread-1" [_thread_blocked, id=374, stack(0x77e60000,0x77eb0000)]
0x77d9c740 JavaThread "SongLoaderCountFiles:2" [_thread_blocked, id=373, stack(0x77cb0000,0x77d00000)]
0x7b670218 JavaThread "qtp9318356-91" [_thread_blocked, id=32431, stack(0x79360000,0x793b0000)]
0x7833f418 JavaThread "qtp9318356-90" [_thread_in_native, id=32430, stack(0x79db0000,0x79e00000)]
0x7b6e7478 JavaThread "MVStore background writer async:/mnt/disk1/share/SongKong/Prefs/Database/Database.mv.db" daemon [_thread_blocked, id=27227, stack(0x79270000,0x792c0000)]
0x7b670cf8 JavaThread "H2 File Lock Watchdog (Socket) async:/mnt/disk1/share/SongKong/Prefs/Database/Database.lock.db" daemon [_thread_blocked, id=27222, stack(0x7a61c000,0x7a66c000)]
0x7b694750 JavaThread "Scheduler-7136558" [_thread_blocked, id=13763, stack(0x78560000,0x785b0000)]
0x7b6786c0 JavaThread "Scheduler-14033461" [_thread_blocked, id=13428, stack(0x785b0000,0x78600000)]
0x7b6717d8 JavaThread "qtp9318356-45-acceptor-0@4cfe9c-ServerConnector@8b18a5{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}" [_thread_in_native, id=13427, stack(0x7904c000,0x7909c000)]
0x789c3e20 JavaThread "Thread-10" [_thread_blocked, id=13405, stack(0x791d0000,0x79220000)]
0x7a0b4f10 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#2" daemon [_thread_blocked, id=13275, stack(0x7a66c000,0x7a6bc000)]
0x7a0b4958 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#1" daemon [_thread_blocked, id=13274, stack(0x79b70000,0x79bc0000)]
0x7a0b43c0 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-HelperThread-#0" daemon [_thread_blocked, id=13273, stack(0x79920000,0x79970000)]
0x7a0feb80 JavaThread "C3P0PooledConnectionPoolManager[identityToken->1hge13oa8796in41o2hlgl|1df6bb7]-AdminTaskTimer" daemon [_thread_blocked, id=13272, stack(0x79c10000,0x79c60000)]
0x7adb1358 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=13146, stack(0x7a7e3000,0x7a833000)]
0xb66f12d8 JavaThread "Service Thread" daemon [_thread_blocked, id=13114, stack(0x7b2b6000,0x7b306000)]
0xb66a1a20 JavaThread "Common-Cleaner" daemon [_thread_blocked, id=13105, stack(0x7b306000,0x7b356000)]
0xb6675c88 JavaThread "Sweeper thread" daemon [_thread_blocked, id=13104, stack(0x7b356000,0x7b3a6000)]
0xb6674978 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=13103, stack(0x7b3a6000,0x7b426000)]
0xb666cdc8 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=13102, stack(0x7b426000,0x7b476000)]
0xb6667688 JavaThread "Finalizer" daemon [_thread_blocked, id=13099, stack(0x7b748000,0x7b798000)]
0xb665a900 JavaThread "Reference Handler" daemon [_thread_blocked, id=13088, stack(0x7b798000,0x7b7e8000)]
0xb6609308 JavaThread "main" [_thread_blocked, id=13037, stack(0xb67d9000,0xb6828000)]
Other Threads:
=>0xb6657d00 VMThread "VM Thread" [stack: 0x7b7e9000,0x7b869000] [id=13074]
0xb66f2a90 WatcherThread [stack: 0x7b236000,0x7b2b6000] [id=13115]
Threads with active compile tasks:
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0xb6608520] Threads_lock - owner thread: 0xb6657d00
[0xb6608930] Heap_lock - owner thread: 0x77a0d790
Heap:
def new generation total 265600K, used 265599K [0x7c000000, 0x8e020000, 0x8ec00000)
eden space 236160K, 100% used [0x7c000000, 0x8a6a0000, 0x8a6a0000)
from space 29440K, 99% used [0x8c360000, 0x8e01fff8, 0x8e020000)
to space 29440K, 0% used [0x8a6a0000, 0x8a6a0000, 0x8c360000)
tenured generation total 589672K, used 298477K [0x8ec00000, 0xb2bda000, 0xb4400000)
the space 589672K, 50% used [0x8ec00000, 0xa0f7b5c0, 0xa0f7b600, 0xb2bda000)
Metaspace used 45545K, capacity 46775K, committed 46976K, reserved 47408K
Card table byte_map: [0x7be3d000,0x7c000000] byte_map_base: 0x7ba5d000
Polling page: 0xb6f30000
回答1:
But surely the question is what does AdaptiveWeightedAverage do ?
If this was the intended question. Then you might have a look into the source.
hotspot/src/share/vm/gc/shared/gcUtil.cpp
// A weighted average maintains a running, weighted average
// of some float value (templates would be handy here if we
// need different types).
//
// The average is adaptive in that we smooth it for the
// initial samples; we don't use the weight until we have
// enough samples for it to be meaningful.
//
// This serves as our best estimate of a future unknown.
//
class AdaptiveWeightedAverage : public CHeapObj<mtGC> {
private:
float _average; // The last computed average
unsigned _sample_count; // How often we've sampled this average
unsigned _weight; // The weight used to smooth the averages
// A higher weight favors the most
// recent data.
bool _is_old; // Has enough historical data
hotspot/src/share/vm/gc/shared/gcUtil.hpp
void AdaptiveWeightedAverage::sample(float new_sample) {
increment_count();
// Compute the new weighted average
float new_avg = compute_adaptive_average(new_sample, average());
set_average(new_avg);
_last_sample = new_sample;
}
and
float AdaptiveWeightedAverage::compute_adaptive_average(float new_sample,
float average) {
// We smooth the samples by not using weight() directly until we've
// had enough data to make it meaningful. We'd like the first weight
// used to be 1, the second to be 1/2, etc until we have
// OLD_THRESHOLD/weight samples.
It's used by the garbage collector.
Maybe somehow related to a problem with available memory. (it's a guess, don't have much experience with Java on ARM platform)
This lead me to this guess max heap -Xmx900m vs system memory 1G.
回答2:
The way out of this was to switch to Java 11 build, this no longer crashes with this error.
来源：https://stackoverflow.com/questions/60095093/java-9-0-1-openjdk-crashed-the-jvm-with-problem-with-adaptiveweightedaverage