in linq why are subsequent calls of IEnumerable.Intersect so much faster

Question

while looking at this question C# Similarities of two arrays it was noted that the initial linq call was significantly slower than subsequent calls. What is being cached th

Answer 1

LINQ heavily uses deferred execution. Unless you enumerate a query, it does not get executed.

Change

 s.Start();
 z= a.Intersect(b);
 s.Stop();

to

 s.Start();
 z= a.Intersect(b).**ToArray**();
 s.Stop();

and please post the new performance results.

a.Intersect(b) represents an expression, independent of the value of a and b. Values of a and b are only used when the expression is evaluated by enumeration.

Answer 2

You are enumerating the result of Intersect() only when you call Count(); that's when the calculation of the intersection actually occurs. The part you're timing is the creation of the enumerable object that represents the future calculation of the intersection.

In addition to the jitting penalty others have noted, the first call to Intersect() might be the first use of a type from System.Core.dll, so you might be looking at the time required to load the IL code into memory, as well.

Answer 3

I would expect the first run of any loop to be slower for three reasons:

1. Code has to be jitted the first time, but not subsequently.
2. If the executable code run is small enough to fit in cache, then it won't have been evicted, and be faster for the CPU to load.
3. If the data is small enough to fit in cache, then it won't have been evicted, and be faster for CPU to load.

Answer 4

Enumerable.Intersect does not do any caching. It is implemented using a HashSet. The first sequence is added to the HashSet. Then the second sequence is removed from the HashSet. The remaining elements in the HashSet is yielded as an enumerable sequence of elements. You will have to actually enumerate the HashSet to pay the cost of creating the HashSet. This implementation is suprisingly efficient even for small collections.

If you see a difference in performance in subsequent calls it is not because Enumerable.Intersect does any caching but probably because you need to "warm up" your benchmark.

Answer 5

JITting System.Enumerable.

Put new List().Intersect(new List()); new System.Diagnostics.Stopwatch().Stop(); as your first line of code and all interations will take the same amount of time.