doseq over a simple lazy seq runs out of heap space

问题

When stress-testing some Clojure code at work, I noticed it runs out of heap space when iterating over large data-sets. I eventually managed to trace the issues back to the combination of Clojure's doseq function, and implementation fo lazy sequences.

This is the minimal code snippet that crashes Clojure by exhausting available heap space:

(doseq [e (take 1000000000 (iterate inc 1))] (identity e))

The documentation for doseq clearly states that it doesn't retain the head of the lazy sequence, so I would expect the memory complexity of the above code to be close to O(1). Is there something I'm missing? What's the Clojure-idiomatic way of iterating over extremely large lazy sequences, if doseq isn't up to the job?

回答1:

When I run this sample I see the memory usage hit 2.0 Gigs so perhaps you are actually just running out of ram.

it sure does take a while to run:

user=> (time (doseq [e (take 1000000000 (iterate inc 1))] (identity e)))
"Elapsed time: 266396.221132 msecs"

form top:

23999 arthur    20   0 4001m 1.2g 5932 S  213 15.3  17:11.35 java                                          
24017 arthur    20   0 3721m 740m 5548 S   88  9.3  13:49.95 java  

来源：https://stackoverflow.com/questions/11932900/doseq-over-a-simple-lazy-seq-runs-out-of-heap-space