Performance benefits of a static empty array instance

Question

It seems common practice to extract a constant empty array return value into a static constant. Like here:

public class NoopParser implements Parser {
    pr         


        

Answer 1

I'm most interested in the actual performance difference between these two idioms in practical, real-world situations. I have no experience in micro-benchmarking (and it is probably not the right tool for such a question) but I gave it a try anyway.

This benchmark models a somewhat more typical, "realistic" setting. The returned array is just looked at and then discarded. No references hanging around, no requirement for reference equality.

One interface, two implementations:

public interface Parser {
    String[] supportedSchemas();
    void parse(String s);
}

public class NoopParserStaticArray implements Parser {
    private static final String[] EMPTY_STRING_ARRAY = new String[0];

    @Override public String[] supportedSchemas() {
        return EMPTY_STRING_ARRAY;
    }

    @Override public void parse(String s) {
        s.codePoints().count();
    }
}

public class NoopParserNewArray implements Parser {
    @Override public String[] supportedSchemas() {
        return new String[0];
    }

    @Override public void parse(String s) {
        s.codePoints().count();
    }
}

And the JMH benchmark:

import org.openjdk.jmh.annotations.Benchmark;

public class EmptyArrayBenchmark {
    private static final Parser NOOP_PARSER_STATIC_ARRAY = new NoopParserStaticArray();
    private static final Parser NOOP_PARSER_NEW_ARRAY = new NoopParserNewArray();

    @Benchmark
    public void staticEmptyArray() {
        Parser parser = NOOP_PARSER_STATIC_ARRAY;
        for (String schema : parser.supportedSchemas()) {
            parser.parse(schema);
        }
    }

    @Benchmark
    public void newEmptyArray() {
        Parser parser = NOOP_PARSER_NEW_ARRAY;
        for (String schema : parser.supportedSchemas()) {
            parser.parse(schema);
        }
    }
}

The result on my machine, Java 1.8.0_51 (HotSpot VM):

Benchmark                              Mode  Cnt           Score          Error  Units
EmptyArrayBenchmark.staticEmptyArray  thrpt   60  3024653836.077 ± 37006870.221  ops/s
EmptyArrayBenchmark.newEmptyArray     thrpt   60  3018798922.045 ± 33953991.627  ops/s
EmptyArrayBenchmark.noop              thrpt   60  3046726348.436 ±  5802337.322  ops/s

There is no significant difference between the two approaches in this case. In fact, they are indistinguishable from the no-op case: apparently the JIT compiler recognises that the returned array is always empty and optimises the loop away entirely!

Piping parser.supportedSchemas() into the black hole instead of looping over it, gives the static array instance approach a ~30% advantage. But they're definitely of the same magnitude:

Benchmark                              Mode  Cnt           Score         Error  Units
EmptyArrayBenchmark.staticEmptyArray  thrpt   60   338971639.355 ±  738069.217  ops/s
EmptyArrayBenchmark.newEmptyArray     thrpt   60   266936194.767 ±  411298.714  ops/s
EmptyArrayBenchmark.noop              thrpt   60  3055609298.602 ± 5694730.452  ops/s

Perhaps in the end the answer is the usual "it depends". I have a hunch that in many practical scenarios, the performance benefit in factoring out the array creation is not significant.

I think it is fair to say that

• if the method contract gives you the freedom to return a new empty array instance every time, and
• unless you need to guard against problematic or pathological usage patterns and/or aim for theoretical max performance,

then returning new String[0] directly is fine.

Personally, I like the expressiveness and concision of return new String[0]; and not having to introduce an extra static field.

---

By some strange coincidence, a month after I wrote this a real performance engineer investigated the problem: see this section in Alexey Shipilёv's blog post 'Arrays of Wisdom of the Ancients':

As expected, the only effect whatsoever can be observed on a very small collection sizes, and this is only a marginal improvement over new Foo[0]. This improvement does not seem to justify caching the array in the grand scheme of things. As a teeny tiny micro-optimization, it might make sense in some tight code, but I wouldn’t care otherwise.

That settles it. I'll take the tick mark and dedicate it to Alexey.

Answer 2

I will go out on a limb and say that the performance benefit, even though using constant is much faster, is not actually relevant; because the software will likely spend a lot more time in doing other things besides returning empty arrays. If the total run-time is even hours a few extra seconds spent in creating an array does not mean much. By the same logic, memory consumption is not relevant either.

The only reason I can think of for doing this is readability.

Answer 3

I benchmarked it using JMH:

private static final String[] EMPTY_STRING_ARRAY = new String[0];

@Benchmark
public void testStatic(Blackhole blackhole) {
    blackhole.consume(EMPTY_STRING_ARRAY);
}

@Benchmark
@Fork(jvmArgs = "-XX:-EliminateAllocations")
public void testStaticEliminate(Blackhole blackhole) {
    blackhole.consume(EMPTY_STRING_ARRAY);
}

@Benchmark
public void testNew(Blackhole blackhole) {
    blackhole.consume(new String[0]);
}

@Benchmark
@Fork(jvmArgs = "-XX:-EliminateAllocations")
public void testNewEliminate(Blackhole blackhole) {
    blackhole.consume(new String[0]);
}

@Benchmark
public void noop(Blackhole blackhole) {
}

Full source code.

Environment (seen after java -jar target/benchmarks.jar -f 1):

# JMH 1.11.2 (released 51 days ago)
# VM version: JDK 1.7.0_75, VM 24.75-b04
# VM invoker: /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
# VM options: <none>
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time

EliminateAllocations was on by default (seen after java -XX:+PrintFlagsFinal -version | grep EliminateAllocations).

Results:

Benchmark                         Mode  Cnt           Score         Error  Units
MyBenchmark.testNewEliminate     thrpt   20    95912464.879 ± 3260948.335  ops/s
MyBenchmark.testNew              thrpt   20   103980230.952 ± 3772243.160  ops/s
MyBenchmark.testStaticEliminate  thrpt   20   206849985.523 ± 4920788.341  ops/s
MyBenchmark.testStatic           thrpt   20   219735906.550 ± 6162025.973  ops/s
MyBenchmark.noop                 thrpt   20  1126421653.717 ± 8938999.666  ops/s

Using a constant was almost two times faster.

Turning off EliminateAllocations slowed things down a tiny bit.

Answer 4

Is the VM not able to roll all empty String arrays into one?

It can't do that, because distinct empty arrays need to compare unequal with ==. Only the programmer can make this optimization.

Contrast this practice with returning an empty String: we're usually perfectly happy writing return "";.

With strings, there is no requirement that distinct string literals produce distinct strings. In every case I know of, two instances of "" will produce the same string object, but maybe there's some weird case with classloaders where that won't happen.