Erratic performance of Arrays.stream().map().sum()

Question

I have chanced upon an instance of exceedingly erratic performance profile of a very simple map/reduce operation on primitive arrays. Here is my jmh benchmark code:

Answer 1

The JIT will first compile the hot loop that is iterating over and operating (map/reduce) on the array elements. This happens quite early on since the array contains 2²⁰ elements.

Later on the JIT compiles the pipeline, most likely inlined within the compiled benchmark method, and due to inlining limits fails to compile it all into one method. It just so happens those inlining limits are reached in the hot loop, and calls to map or sum are not inlined, so the hot loop is unintentionally "de-optimized".

Use the options -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompilation -XX:+PrintInlining when running the benchmark and early on you should see output like the following:

   1202  487 %     4       java.util.Spliterators$IntArraySpliterator::forEachRemaining @ 49 (68 bytes)
                              @ 53   java.util.stream.IntPipeline$3$1::accept (23 bytes)   inline (hot)
                               \-> TypeProfile (1186714/1186714 counts) = java/util/stream/IntPipeline$3$1
                                @ 12   test.Measure$$Lambda$2/1745776415::applyAsInt (9 bytes)   inline (hot)
                                 \-> TypeProfile (1048107/1048107 counts) = test/Measure$$Lambda$2
                                  @ 5   test.Measure::lambda$setup$1 (4 bytes)   inline (hot)
                                @ 17   java.util.stream.ReduceOps$5ReducingSink::accept (19 bytes)   inline (hot)
                                 \-> TypeProfile (1048107/1048107 counts) = java/util/stream/ReduceOps$5ReducingSink
                                  @ 10   java.util.stream.IntPipeline$$Lambda$3/1779653790::applyAsInt (6 bytes)   inline (hot)
                                   \-> TypeProfile (1048064/1048064 counts) = java/util/stream/IntPipeline$$Lambda$3
                                    @ 2   java.lang.Integer::sum (4 bytes)   inline (hot)

That is the hot loop getting compiled. (The % means it is On Stack Replaced, or OSR'ed)

Later on further compilation of the stream pipeline occurs (at i suspect ~10,000 iterations of the benchmark method, but I have not verified):

                          @ 16   java.util.stream.IntPipeline::sum (11 bytes)   inline (hot)
                           \-> TypeProfile (5120/5120 counts) = java/util/stream/IntPipeline$3
                            @ 2   java.lang.invoke.LambdaForm$MH/1279902262::linkToTargetMethod (8 bytes)   force inline by annotation
                              @ 4   java.lang.invoke.LambdaForm$MH/1847865997::identity (18 bytes)   force inline by annotation
                                @ 14   java.lang.invoke.LambdaForm$DMH/2024969684::invokeStatic_L_L (14 bytes)   force inline by annotation
                                  @ 1   java.lang.invoke.DirectMethodHandle::internalMemberName (8 bytes)   force inline by annotation
                                  @ 10   sun.invoke.util.ValueConversions::identity (2 bytes)   inline (hot)
                            @ 7   java.util.stream.IntPipeline::reduce (16 bytes)   inline (hot)
                              @ 3   java.util.stream.ReduceOps::makeInt (18 bytes)   inline (hot)
                                @ 1   java.util.Objects::requireNonNull (14 bytes)   inline (hot)
                                @ 14   java.util.stream.ReduceOps$5::<init> (16 bytes)   inline (hot)
                                  @ 12   java.util.stream.ReduceOps$ReduceOp::<init> (10 bytes)   inline (hot)
                                    @ 1   java.lang.Object::<init> (1 bytes)   inline (hot)
                              @ 6   java.util.stream.AbstractPipeline::evaluate (94 bytes)   inline (hot)
                                @ 50   java.util.stream.AbstractPipeline::isParallel (8 bytes)   inline (hot)
                                @ 80   java.util.stream.TerminalOp::getOpFlags (2 bytes)   inline (hot)
                                 \-> TypeProfile (5122/5122 counts) = java/util/stream/ReduceOps$5
                                @ 85   java.util.stream.AbstractPipeline::sourceSpliterator (163 bytes)   inline (hot)
                                  @ 79   java.util.stream.AbstractPipeline::isParallel (8 bytes)   inline (hot)
                                @ 88   java.util.stream.ReduceOps$ReduceOp::evaluateSequential (18 bytes)   inline (hot)
                                  @ 2   java.util.stream.ReduceOps$5::makeSink (5 bytes)   inline (hot)
                                    @ 1   java.util.stream.ReduceOps$5::makeSink (16 bytes)   inline (hot)
                                      @ 12   java.util.stream.ReduceOps$5ReducingSink::<init> (15 bytes)   inline (hot)
                                        @ 11   java.lang.Object::<init> (1 bytes)   inline (hot)
                                  @ 6   java.util.stream.AbstractPipeline::wrapAndCopyInto (18 bytes)   inline (hot)
                                    @ 3   java.util.Objects::requireNonNull (14 bytes)   inline (hot)
                                    @ 9   java.util.stream.AbstractPipeline::wrapSink (37 bytes)   inline (hot)
                                      @ 1   java.util.Objects::requireNonNull (14 bytes)   inline (hot)
                                      @ 23   java.util.stream.IntPipeline$3::opWrapSink (10 bytes)   inline (hot)
                                       \-> TypeProfile (4868/4868 counts) = java/util/stream/IntPipeline$3
                                        @ 6   java.util.stream.IntPipeline$3$1::<init> (11 bytes)   inline (hot)
                                          @ 7   java.util.stream.Sink$ChainedInt::<init> (16 bytes)   inline (hot)
                                            @ 1   java.lang.Object::<init> (1 bytes)   inline (hot)
                                            @ 6   java.util.Objects::requireNonNull (14 bytes)   inline (hot)
                                    @ 13   java.util.stream.AbstractPipeline::copyInto (53 bytes)   inline (hot)
                                      @ 1   java.util.Objects::requireNonNull (14 bytes)   inline (hot)
                                      @ 9   java.util.stream.AbstractPipeline::getStreamAndOpFlags (5 bytes)   accessor
                                      @ 12   java.util.stream.StreamOpFlag::isKnown (19 bytes)   inline (hot)
                                      @ 20   java.util.Spliterator::getExactSizeIfKnown (25 bytes)   inline (hot)
                                       \-> TypeProfile (4870/4870 counts) = java/util/Spliterators$IntArraySpliterator
                                        @ 1   java.util.Spliterators$IntArraySpliterator::characteristics (5 bytes)   accessor
                                        @ 19   java.util.Spliterators$IntArraySpliterator::estimateSize (11 bytes)   inline (hot)
                                      @ 25   java.util.stream.Sink$ChainedInt::begin (11 bytes)   inline (hot)
                                       \-> TypeProfile (4870/4870 counts) = java/util/stream/IntPipeline$3$1
                                        @ 5   java.util.stream.ReduceOps$5ReducingSink::begin (9 bytes)   inline (hot)
                                         \-> TypeProfile (4871/4871 counts) = java/util/stream/ReduceOps$5ReducingSink
                                      @ 32   java.util.Spliterator$OfInt::forEachRemaining (53 bytes)   inline (hot)
                                        @ 12   java.util.Spliterators$IntArraySpliterator::forEachRemaining (68 bytes)   inline (hot)
                                          @ 53   java.util.stream.IntPipeline$3$1::accept (23 bytes)   inline (hot)
                                            @ 12   test.Measure$$Lambda$2/1745776415::applyAsInt (9 bytes)   inline (hot)
                                             \-> TypeProfile (1048107/1048107 counts) = test/Measure$$Lambda$2
                                              @ 5   test.Measure::lambda$setup$1 (4 bytes)   inlining too deep
                                            @ 17   java.util.stream.ReduceOps$5ReducingSink::accept (19 bytes)   inline (hot)
                                             \-> TypeProfile (1048107/1048107 counts) = java/util/stream/ReduceOps$5ReducingSink
                                              @ 10   java.util.stream.IntPipeline$$Lambda$3/1779653790::applyAsInt (6 bytes)   inlining too deep
                                               \-> TypeProfile (1048064/1048064 counts) = java/util/stream/IntPipeline$$Lambda$3
                                          @ 53   java.util.stream.IntPipeline$3$1::accept (23 bytes)   inline (hot)
                                            @ 12   test.Measure$$Lambda$2/1745776415::applyAsInt (9 bytes)   inline (hot)
                                             \-> TypeProfile (1048107/1048107 counts) = test/Measure$$Lambda$2
                                              @ 5   test.Measure::lambda$setup$1 (4 bytes)   inlining too deep
                                            @ 17   java.util.stream.ReduceOps$5ReducingSink::accept (19 bytes)   inline (hot)
                                             \-> TypeProfile (1048107/1048107 counts) = java/util/stream/ReduceOps$5ReducingSink
                                              @ 10   java.util.stream.IntPipeline$$Lambda$3/1779653790::applyAsInt (6 bytes)   inlining too deep
                                               \-> TypeProfile (1048064/1048064 counts) = java/util/stream/IntPipeline$$Lambda$3
                                      @ 38   java.util.stream.Sink$ChainedInt::end (10 bytes)   inline (hot)
                                        @ 4   java.util.stream.Sink::end (1 bytes)   inline (hot)
                                         \-> TypeProfile (5120/5120 counts) = java/util/stream/ReduceOps$5ReducingSink
                                  @ 12   java.util.stream.ReduceOps$5ReducingSink::get (5 bytes)   inline (hot)
                                    @ 1   java.util.stream.ReduceOps$5ReducingSink::get (8 bytes)   inline (hot)
                                      @ 4   java.lang.Integer::valueOf (32 bytes)   inline (hot)
                                        @ 28   java.lang.Integer::<init> (10 bytes)   inline (hot)
                                          @ 1   java.lang.Number::<init> (5 bytes)   inline (hot)
                                            @ 1   java.lang.Object::<init> (1 bytes)   inline (hot)
                              @ 12   java.lang.Integer::intValue (5 bytes)   accessor

Note the "inlining too deep" that occurs for methods in the hot loop.

Even later on the generated JMH measurement loop is compiled:

  26857  685       3       test.generated.Measure_multiply::multiply_avgt_jmhLoop (55 bytes)
                              @ 7   java.lang.System::nanoTime (0 bytes)   intrinsic
                              @ 16   test.Measure::multiply (23 bytes)
                                @ 4   java.util.Arrays::stream (8 bytes)
                                  @ 4   java.util.Arrays::stream (11 bytes)
                                    @ 3   java.util.Arrays::spliterator (10 bytes)
                                      @ 6   java.util.Spliterators::spliterator (25 bytes)   callee is too large
                                    @ 7   java.util.stream.StreamSupport::intStream (14 bytes)
                                      @ 6   java.util.stream.StreamOpFlag::fromCharacteristics (37 bytes)   callee is too large
                                      @ 10   java.util.stream.IntPipeline$Head::<init> (8 bytes)
                                        @ 4   java.util.stream.IntPipeline::<init> (8 bytes)
                                          @ 4   java.util.stream.AbstractPipeline::<init> (55 bytes)   callee is too large
                                @ 11   java.util.stream.IntPipeline::map (26 bytes)
                                  @ 1   java.util.Objects::requireNonNull (14 bytes)
                                    @ 8   java.lang.NullPointerException::<init> (5 bytes)   don't inline Throwable constructors
                                  @ 22   java.util.stream.IntPipeline$3::<init> (20 bytes)
                                    @ 16   java.util.stream.IntPipeline$StatelessOp::<init> (29 bytes)   callee is too large
                                @ 16   java.util.stream.IntPipeline::sum (11 bytes)
                                  @ 2   java.lang.invoke.LambdaForm$MH/1279902262::linkToTargetMethod (8 bytes)   force inline by annotation
                                    @ 4   java.lang.invoke.LambdaForm$MH/1847865997::identity (18 bytes)   force inline by annotation
                                      @ 14   java.lang.invoke.LambdaForm$DMH/2024969684::invokeStatic_L_L (14 bytes)   force inline by annotation
                                        @ 1   java.lang.invoke.DirectMethodHandle::internalMemberName (8 bytes)   force inline by annotation
                                        @ 10   sun.invoke.util.ValueConversions::identity (2 bytes)
                                  @ 7   java.util.stream.IntPipeline::reduce (16 bytes)
                                    @ 3   java.util.stream.ReduceOps::makeInt (18 bytes)
                                      @ 1   java.util.Objects::requireNonNull (14 bytes)
                                      @ 14   java.util.stream.ReduceOps$5::<init> (16 bytes)
                                        @ 12   java.util.stream.ReduceOps$ReduceOp::<init> (10 bytes)
                                          @ 1   java.lang.Object::<init> (1 bytes)
                                    @ 6   java.util.stream.AbstractPipeline::evaluate (94 bytes)   callee is too large
                                    @ 12   java.lang.Integer::intValue (5 bytes)

Note that there is no attempt to inline the whole stream pipeline, it stops well before it reached the hot loop, see "callee is too large", thereby re-optimizing the hot loop.

The inline limit can be increased to avoid such behaviour, for example -XX:MaxInlineLevel=12.