Returning IEnumerable vs. IQueryable

Question

What is the difference between returning IQueryable vs. IEnumerable, when should one be preferred over the other?

Answer 1

In general you want to preserve the original static type of the query until it matters.

For this reason, you can define your variable as 'var' instead of either IQueryable<> or IEnumerable<> and you will know that you are not changing the type.

If you start out with an IQueryable<>, you typically want to keep it as an IQueryable<> until there is some compelling reason to change it. The reason for this is that you want to give the query processor as much information as possible. For example, if you're only going to use 10 results (you've called Take(10)) then you want SQL Server to know about that so that it can optimize its query plans and send you only the data you'll use.

A compelling reason to change the type from IQueryable<> to IEnumerable<> might be that you are calling some extension function that the implementation of IQueryable<> in your particular object either cannot handle or handles inefficiently. In that case, you might wish to convert the type to IEnumerable<> (by assigning to a variable of type IEnumerable<> or by using the AsEnumerable extension method for example) so that the extension functions you call end up being the ones in the Enumerable class instead of the Queryable class.

Answer 2

I recently ran into an issue with IEnumerable v. IQueryable. The algorithm being used first performed an IQueryable query to obtain a set of results. These were then passed to a foreach loop, with the items instantiated as an Entity Framework (EF) class. This EF class was then used in the from clause of a Linq to Entity query, causing the result to be IEnumerable.

I'm fairly new to EF and Linq for Entities, so it took a while to figure out what the bottleneck was. Using MiniProfiling, I found the query and then converted all of the individual operations to a single IQueryable Linq for Entities query. The IEnumerable took 15 seconds and the IQueryable took 0.5 seconds to execute. There were three tables involved and, after reading this, I believe that the IEnumerable query was actually forming a three table cross-product and filtering the results.

Try to use IQueryables as a rule-of-thumb and profile your work to make your changes measurable.

Answer 3

I would like to clarify a few things due to seemingly conflicting responses (mostly surrounding IEnumerable).

(1) IQueryable extends the IEnumerable interface. (You can send an IQueryable to something which expects IEnumerable without error.)

(2) Both IQueryable and IEnumerable LINQ attempt lazy loading when iterating over the result set. (Note that implementation can be seen in interface extension methods for each type.)

In other words, IEnumerables are not exclusively "in-memory". IQueryables are not always executed on the database. IEnumerable must load things into memory (once retrieved, possibly lazily) because it has no abstract data provider. IQueryables rely on an abstract provider (like LINQ-to-SQL), although this could also be the .NET in-memory provider.

Sample use case

(a) Retrieve list of records as IQueryable from EF context. (No records are in-memory.)

(b) Pass the IQueryable to a view whose model is IEnumerable. (Valid. IQueryable extends IEnumerable.)

(c) Iterate over and access the data set's records, child entities and properties from the view. (May cause exceptions!)

Possible Issues

(1) The IEnumerable attempts lazy loading and your data context is expired. Exception thrown because provider is no longer available.

(2) Entity Framework entity proxies are enabled (the default), and you attempt to access a related (virtual) object with an expired data context. Same as (1).

(3) Multiple Active Result Sets (MARS). If you are iterating over the IEnumerable in a foreach( var record in resultSet ) block and simultaneously attempt to access record.childEntity.childProperty, you may end up with MARS due to lazy loading of both the data set and the relational entity. This will cause an exception if it is not enabled in your connection string.

Solution

• I have found that enabling MARS in the connection string works unreliably. I suggest you avoid MARS unless it is well-understood and explicitly desired.

Execute the query and store results by invoking resultList = resultSet.ToList() This seems to be the most straightforward way of ensuring your entities are in-memory.

In cases where the you are accessing related entities, you may still require a data context. Either that, or you can disable entity proxies and explicitly Include related entities from your DbSet.

Answer 4

A lot has been said previously, but back to the roots, in a more technical way:

1. IEnumerable is a collection of objects in memory that you can enumerate - an in-memory sequence that makes it possible to iterate through (makes it way easy for within foreach loop, though you can go with IEnumerator only). They reside in the memory as is.
2. IQueryable is an expression tree that will get translated into something else at some point with ability to enumerate over the final outcome. I guess this is what confuses most people.

They obviously have different connotations.

IQueryable represents an expression tree (a query, simply) that will be translated to something else by the underlying query provider as soon as release APIs are called, like LINQ aggregate functions (Sum, Count, etc.) or ToList[Array, Dictionary,...]. And IQueryable objects also implement IEnumerable, IEnumerable<T> so that if they represent a query the result of that query could be iterated. It means IQueryable don't have to be queries only. The right term is they are expression trees.

Now how those expressions are executed and what they turn to is all up to so called query providers (expression executors we can think them of).

In the Entity Framework world (which is that mystical underlying data source provider, or the query provider) IQueryable expressions are translated into native T-SQL queries. Nhibernate does similar things with them. You can write your own one following the concepts pretty well described in LINQ: Building an IQueryable Provider link, for example, and you might want to have a custom querying API for your product store provider service.

So basically, IQueryable objects are getting constructed all the way long until we explicitly release them and tell the system to rewrite them into SQL or whatever and send down the execution chain for onward processing.

As if to deferred execution it's a LINQ feature to hold up the expression tree scheme in the memory and send it into the execution only on demand, whenever certain APIs are called against the sequence (the same Count, ToList, etc.).

The proper usage of both heavily depends on the tasks you're facing for the specific case. For the well-known repository pattern I personally opt for returning IList, that is IEnumerable over Lists (indexers and the like). So it is my advice to use IQueryable only within repositories and IEnumerable anywhere else in the code. Not saying about the testability concerns that IQueryable breaks down and ruins the separation of concerns principle. If you return an expression from within repositories consumers may play with the persistence layer as they would wish.

A little addition to the mess :) (from a discussion in the comments)) None of them are objects in memory since they're not real types per se, they're markers of a type - if you want to go that deep. But it makes sense (and that's why even MSDN put it this way) to think of IEnumerables as in-memory collections whereas IQueryables as expression trees. The point is that the IQueryable interface inherits the IEnumerable interface so that if it represents a query, the results of that query can be enumerated. Enumeration causes the expression tree associated with an IQueryable object to be executed. So, in fact, you can't really call any IEnumerable member without having the object in the memory. It will get in there if you do, anyways, if it's not empty. IQueryables are just queries, not the data.

Answer 5

Both will give you deferred execution, yes.

As for which is preferred over the other, it depends on what your underlying datasource is.

Returning an IEnumerable will automatically force the runtime to use LINQ to Objects to query your collection.

Returning an IQueryable (which implements IEnumerable, by the way) provides the extra functionality to translate your query into something that might perform better on the underlying source (LINQ to SQL, LINQ to XML, etc.).

Answer 6

The top answer is good but it doesn't mention expression trees which explain "how" the two interfaces differ. Basically, there are two identical sets of LINQ extensions. Where(), Sum(), Count(), FirstOrDefault(), etc all have two versions: one that accepts functions and one that accepts expressions.

• The IEnumerable version signature is: Where(Func<Customer, bool> predicate)

• The IQueryable version signature is: Where(Expression<Func<Customer, bool>> predicate)

You've probably been using both of those without realizing it because both are called using identical syntax:

e.g. Where(x => x.City == "<City>") works on both IEnumerable and IQueryable

• When using Where() on an IEnumerable collection, the compiler passes a compiled function to Where()

• When using Where() on an IQueryable collection, the compiler passes an expression tree to Where(). An expression tree is like the reflection system but for code. The compiler converts your code into a data structure that describes what your code does in a format that's easily digestible.

Why bother with this expression tree thing? I just want Where() to filter my data. The main reason is that both the EF and Linq2SQL ORMs can convert expression trees directly into SQL where your code will execute much faster.

Oh, that sounds like a free performance boost, should I use AsQueryable() all over the place in that case? No, IQueryable is only useful if the underlying data provider can do something with it. Converting something like a regular List to IQueryable will not give you any benefit.