How to do fast data deserialization in Haskell

Question

Benchmarking shows that the cereal library takes 100x longer to deserialize a data structure of mine (detailed below) than it takes to read the same data off the dr

Answer 1

Ok. To answer this with the summary of the advice. For fast deserialization of data:

• Use cereal (strict bytestring output) or binary (lazy bytetring output)
• Make sure you're compiling with -O2, as these libraries rely on inlining to remove overhead
• Use dense data types, such as replacing a polymorphic tuple with an unpacked, specialized form.
• Avoid converting data types to lists to serialize them. If you have bytestrings, this is taken care of. For unpacked array types, you usually will get very fast IO, but worth double checking the instances
• You may be able to use mmap'd IO
• For double-heavy data consider a more efficient double reader.
• Use modern array and container types tuned for performance, with more recent GHC versions.