Global / local environment affects Haskell's Criterion benchmarks results

Question

We\'re benchmarking some Haskell code in our company and we\'ve just hit a very strange case. Here is a code, which benchmarks the same thing 2 times. The former one uses an

Answer 1

As with your presumably-coworker's question, I can not reproduce this issue. Note I am using GHC 8.2.2 and criterion 1.3.0.0

benchmarking env per all runs/10e7
time                 18.94 ms   (18.71 ms .. 19.22 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 19.59 ms   (19.39 ms .. 19.99 ms)
std dev              618.3 μs   (379.7 μs .. 952.8 μs)

benchmarking env per all runs/10e8
time                 192.0 ms   (189.5 ms .. 194.9 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 191.8 ms   (190.3 ms .. 193.1 ms)
std dev              1.778 ms   (1.088 ms .. 2.457 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking env per every run/10e7
time                 18.97 ms   (18.38 ms .. 19.62 ms)
                     0.999 R²   (0.996 R² .. 1.000 R²)
mean                 18.98 ms   (18.83 ms .. 19.35 ms)
std dev              298.8 μs   (25.39 μs .. 391.3 μs)
variance introduced by outliers: 14% (moderately inflated)

benchmarking env per every run/10e8
time                 194.0 ms   (182.0 ms .. 211.6 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 192.0 ms   (189.0 ms .. 193.9 ms)
std dev              2.850 ms   (0.0 s .. 3.261 ms)
variance introduced by outliers: 19% (moderately inflated)

./p  18.32s user 1.16s system 99% cpu 19.531 total
% ghc-pkg list criterion
...
    criterion-1.3.0.0

I notice my version of criterion was uploaded after the bug @jberryman pointed to was fixed, so perhaps this is the difference.