Compiling very large constants with GHC

Question

Today I asked GHC to compile an 8MB Haskell source file. GHC thought about it for about 6 minutes, swallowing almost 2GB of RAM, and then finally gave up with an out-of-memory e

Answer 1

Your best bet is probably to compile a string representation of your value into the executable. To do this in a clean manner, please refer to my answer in a previous question.

To use it, simply store your expression in myExpression.exp and do read [litFile|myExpression.exp|] with the QuasiQuotes extension enabled, and the expression will be "stored as a string literal" in the executable.

---

I tried doing something similar for storing actual constants, but it fails for the same reason that embedding the value in a .hs file would. My attempt was:

Verbatim.hs:

module Verbatim where

import Language.Haskell.TH
import Language.Haskell.TH.Quote
import Language.Haskell.Meta.Parse

readExp :: String -> Q Exp
readExp = either fail return . parseExp

verbatim :: QuasiQuoter
verbatim = QuasiQuoter { quoteExp = readExp }

verbatimFile :: QuasiQuoter
verbatimFile = quoteFile verbatim

Test program:

{-# LANGUAGE QuasiQuotes #-}
module Main (main) where

import Verbatim

main :: IO ()
main = print [verbatimFile|test.exp|]

This program works for small test.exp files, but fails already at about 2MiB on this computer.

Answer 2

There's a simple solution — your literal should have type ByteString. See https://github.com/litherum/publicsuffixlist/pull/1 for details.