How to make my Haskell program faster? Comparison with C

Question

I\'m working on an implementation of one of the SHA3 candidates, JH. I\'m at the point where the algorithm pass all KATs (Known Answer Tests) provided by NIST, and have also mad

Answer 1

Ok, so I thought I would chime in with an update of what I have done and the results obtained thus far. Changes made:

• Switched from Array to UnboxedArray (made Word128 an instance type)
• Used UnboxedArray + fold in e8 instead of lists and (prelude) fold
• Used unsafeIndex instead of !
• Changed type of Block1024 to a real datatype (similiar to Block512), and unpacked its arguments
• Updated GHC to version 7.2.1 on Arch Linux, thus fixing the problem with compiling via C or LLVM
• Switched mod to rem in some places, but NOT in roundFunction. When I do it there, the compile time suddenly takes an awful lot of time, and the run time becomes 10 times slower! Does anyone know why that may be? It is only happening with GHC-7.2.1, not GHC-7.0.3

I compile with the following options:

ghc-7.2.1 --make -O2 -funbox-strict-fields main.hs ./Tests/testframe.hs -fvia-C -optc-O2

And the results? Roughly 50 % reduction in time. On an input of ~107 MB, the code now use 3 minutes as compared to the previous 6-7 minutes. The C version uses 42 seconds.

Things I tried, but which didn't result in better performance:

• Unrolled the e8 function like this:

  e8 !h = go h 0

  where go !x !n

        | n == 42   = x
        | otherwise = go h' (n + 1)
        where !h' = roundFunction x n
  

• Tried breaking up the swapN functions to use the underlying Word64' directly:

  swap1 (W xh hl) =

       shiftL (W (xh .&. 0x5555555555555555) (xl .&. 0x5555555555555555)) 1 
       .|. 
       shiftR (W (xh .&. 0xaaaaaaaaaaaaaaaa) (xl .&. 0xaaaaaaaaaaaaaaaa)) 1 
  

• Tried using the LLVM backend

All of these attempts gave worse performance than what I have currently. I don't know if thats because I'm doing it wrong (especially the unrolling of e8), or because they just are worse options.

Still I have some new questions with these new tweaks.

1. Suddenly I have gotten this peculiar bump in memory usage. Take a look at following heap profiles:

   Why has this happened? Is it because of the UnboxedArray? And what does SYSTEM mean?

2. When I compile via C I get the following warning:

   Warning: The -fvia-C flag does nothing; it will be removed in a future GHC release

   Is this true? Why then, do I see better performance using it, rather than not?