List comprehension vs high-order functions in F#

Question

I come from SML background and feel quite comfortable with high-order functions. But I don\'t really get the idea of list comprehension. Is there any situation where list co

Answer 1

Choosing between comprehensions and higher-order functions is mostly a matter of style. I think that comprehensions are sometimes more readable, but that's just a personal preference. Note that the cartesian function could be written more elegantly like this:

let rec cartesian = function  
  | [] -> [[]]  
  | L::Ls -> 
     [ for C in cartesian Ls do for x in L do yield x::C ]

The interesting case is when writing recursive functions. If you use sequences (and sequence comprehensions), they remove some unnecessary allocation of temporary lists and if you use yield! in a tail-call position, you can also avoid stack overflow exceptions:

let rec nums n = 
  if n = 100000 then []
  else n::(nums (n+1))
// throws StackOverflowException
nums 0 

let rec nums n = seq {
  if n < 100000 then
    yield n
    yield! nums (n+1) }
// works just fine
nums 0 |> List.ofSeq 

This is quite an interesting pattern, because it cannot be written in the same way using lists. When using lists, you cannot return some element and then make a recursive call, because it corresponds to n::(nums ...), which is not tail-recursive.

Answer 2

Looking at the generated code in ILSpy, you can see that list comprehensions are compiled to state machines (like methods using yield return in C#), then passed to something like List.ofSeq. Higher-order functions, on the other hand, are hand-coded, and frequently use mutable state or other imperative constructs to be as efficient as possible. As is often the case, the general-purpose mechanism is more expensive.

So, to answer your question, if performance is critical there is usually a higher-order function specific to your problem that should be preferred.

Answer 3

Adding to Tomas Petricek's answer. You can make the list version tail recursive.

let nums3 n =
    let rec nums3internal acc n = 
        if n = 100000 then
            acc
        else
            nums3internal (n::acc) (n+1) //Tail Call Optimization possible

    nums3internal [] n |> List.rev

nums3 0

With the added benefit of a considerable speedup. At least when I measured with the stopwatch tool I get. (nums2 being the algorithm using Seq).

Nums2 takes 81.225500ms
Nums3 takes 4.948700ms

For higher numbers this advantage shrinks, because List.rev is inefficient. E.g. for 10000000 I get:

Nums2 takes 11054.023900ms
Nums3 takes 8256.693100ms