How can I write a clean Repository without exposing IQueryable to the rest of my application?

Question

So, I\'ve read all the Q&A\'s here on SO regarding the subject of whether or not to expose IQueryable to the rest of your project or not (see here, and here), and I\'ve ulti

Answer 1

I ended up creating two sets of methods, ones that return IEnumerable (in your case IQueryable), and ones that return Collection (pull the content before sending it out of the repository.)

This allows me to do both build ad hoc queries in Services outside the repository as well as use Repository methods directly returning side effect resistant Collections. In other words, joining two Repository Entities together results in one select query, instead of one select query for each entity found.

I imagine you could set your protection level to keep truly bad things from happening.

Answer 2

hmm.. I solved this in many ways depending on the type of ORM i use.
The main idea is to have one repository base class and one query method that takes so many parameters indicating all possible where/orderby/expand|include/paging/etc options.

Here is a quick and dirty sample using LINQ to NHibernate (of course the entire repository should be implementation detail):

public class RepositoryBase
    {
        private ISession Session;

        public RepositoryBase()
        {
            Session = SessionPlaceHolder.Session;
        }



        public TEntity[] GetPaged<TEntity>(IEnumerable<Expression<Func<TEntity, bool>>> filters,
            IEnumerable<Expression<Func<TEntity, object>>> relatedObjects,
            IEnumerable<Expression<Func<TEntity, object>>> orderCriterias,
            IEnumerable<Expression<Func<TEntity, object>>> descOrderCriterias,
            int pageNumber, int pageSize, out int totalPages)
        {
            INHibernateQueryable<TEntity> nhQuery = Session.Linq<TEntity>();

            if (relatedObjects != null)
                foreach (var relatedObject in relatedObjects)
                {
                    if (relatedObject == null) continue;
                    nhQuery = nhQuery.Expand(relatedObject);
                }

            IQueryable<TEntity> query = nhQuery;

            if (filters != null)
                foreach (var filter in filters)
                {
                    if (filter == null) continue;
                    query = query.Where(filter);
                }

            bool pagingEnabled = pageSize > 0;

            if (pagingEnabled)
                totalPages = (int) Math.Ceiling((decimal) query.Count()/(decimal) pageSize);
            else
                totalPages = 1;

            if (orderCriterias != null)
                foreach (var orderCriteria in orderCriterias)
                {
                    if (orderCriteria == null) continue;
                    query = query.OrderBy(orderCriteria);
                }

            if (descOrderCriterias != null) 
                foreach (var descOrderCriteria in descOrderCriterias)
                {
                    if (descOrderCriteria == null) continue;
                    query = query.OrderByDescending(descOrderCriteria);
                }

            if (pagingEnabled)
                query = query.Skip(pageSize*(pageNumber - 1)).Take(pageSize);

            return query.ToArray();
        }
    }

Normally you'll want to add many chaining overloads as shortcuts when you don't need paging for example, etc..

Here is another dirty one. Sorry I'm not sure if I can expose the final ones. Those were drafts and are OK to show:

using Context = Project.Services.Repositories.EntityFrameworkContext;
using EntitiesContext = Project.Domain.DomainSpecificEntitiesContext;    
namespace Project.Services.Repositories
{
    public class EntityFrameworkRepository : IRepository
    {
        #region IRepository Members

        public bool TryFindOne<T>(Expression<Func<T, bool>> filter, out T result)
        {
            result = Find(filter, null).FirstOrDefault();

            return !Equals(result, default(T));
        }

        public T FindOne<T>(Expression<Func<T, bool>> filter)
        {
            T result;
            if (TryFindOne(filter, out result))
                return result;

            return default(T);
        }

        public IList<T> Find<T>() where T : class, IEntityWithKey
        {
            int count;
            return new List<T>(Find<T>(null, null, 0, 0, out count));
        }

        public IList<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort)
        {
            int count;
            return new List<T>(Find(filter, sort, 0, 0, out count));
        }

        public IEnumerable<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort, int pageSize,
                                      int pageNumber, out int count)
        {
            return ExecuteQuery(filter, sort, pageSize, pageNumber, out count) ?? new T[] {};
        }

        public bool Save<T>(T entity)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            EntityKey key = context.CreateEntityKey(GetEntitySetName(entity.GetType()), entity);

            object originalItem;
            if (context.TryGetObjectByKey(key, out originalItem))
            {
                context.ApplyPropertyChanges(key.EntitySetName, entity);
            }
            else
            {
                context.AddObject(GetEntitySetName(entity.GetType()), entity);
                //Attach(context, entity);
            }

            return context.SaveChanges() > 0;
        }

        public bool Delete<T>(Expression<Func<T, bool>> filter)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            int numberOfObjectsFound = 0;
            foreach (T entity in context.CreateQuery<T>(GetEntitySetName(typeof (T))).Where(filter))
            {
                context.DeleteObject(entity);
                ++numberOfObjectsFound;
            }

            return context.SaveChanges() >= numberOfObjectsFound;
        }

        #endregion

        protected IEnumerable<T> ExecuteQuery<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort,
                                                 int pageSize, int pageNumber,
                                                 out int count)
        {
            IEnumerable<T> result;

            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            ObjectQuery<T> originalQuery = CreateQuery<T>(context);
            IQueryable<T> query = originalQuery;

            if (filter != null)
                query = query.Where(filter);

            if (sort != null)
                query = query.OrderBy(sort);

            if (pageSize > 0)
            {
                int pageIndex = pageNumber > 0 ? pageNumber - 1 : 0;
                query = query.Skip(pageIndex).Take(pageSize);

                count = query.Count();
            }
            else 
                count = -1;


            result = ExecuteQuery(context, query);

            //if no paging total count is count of the entire result set
            if (count == -1) count = result.Count();

            return result;
        }

        protected internal event Action<ObjectContext, IEnumerable> EntitiesFound;

        protected void OnEntitiesFound<T>(ObjectContext context, params T[] entities)
        {
            if (EntitiesFound != null && entities != null && entities.Length > 0)
            {
                EntitiesFound(context, entities);
            }
        }

        //Allowing room for system-specific-requirement extensibility
        protected Action<IEnumerable> ItemsFound;

        protected IEnumerable<T> ExecuteQuery<T>(ObjectContext context, IQueryable<T> query)
        {
            IEnumerable<T> result = null;

            if (query is ObjectQuery)
            {
                var objectQuery = (ObjectQuery<T>) query;

                objectQuery.EnablePlanCaching = false;
                objectQuery.MergeOption = MergeOption.PreserveChanges;

                result = new List<T>(objectQuery);

                if (ItemsFound != null)
                    ItemsFound(result);

                return result;
            }

            return result;
        }

        internal static RelationshipManager GetRelationshipManager(object entity)
        {
            var entityWithRelationships = entity as IEntityWithRelationships;
            if (entityWithRelationships != null)
            {
                return entityWithRelationships.RelationshipManager;
            }

            return null;
        }


        protected ObjectQuery<T> CreateQuery<T>(ObjectContext context)
        {
            ObjectQuery<T> query = context.CreateQuery<T>(GetEntitySetName(typeof (T)));
            query = this.AggregateEntities(query);
            return query;
        }

        protected virtual ObjectQuery<T> AggregateEntities<T>(ObjectQuery<T> query)
        {
            return query;
        }

        private static string GetEntitySetName(Type entityType)
        {
            return string.Format("{0}Set", entityType.Name);
        }
    }

    public class EntityFrameworkContext
    {
        private const string CtxKey = "ctx";

        private bool contextInitialized
        {
            get { return HttpContext.Current.Items[CtxKey] != null;  }
        }

        public EntitiesContext Context
        {
            get
            {
                if (contextInitialized == false)
                {
                    HttpContext.Current.Items[CtxKey] = new EntitiesContext(ConfigurationManager.ConnectionStrings["CoonectionStringName"].ToString());
                }

                return (EntitiesContext)HttpContext.Current.Items[CtxKey];
            }
        }

        public void TrulyDispose()
        {
            if (contextInitialized)
            {
                Context.Dispose();
                HttpContext.Current.Items[CtxKey] = null;
            }
        }
    }

    internal static class EntityFrameworkExtensions
    {
        internal static ObjectQuery<T> Include<T>(this ObjectQuery<T> query,
                                                  Expression<Func<T, object>> propertyToInclude)
        {
            string include = string.Join(".", propertyToInclude.Body.ToString().Split('.').Skip(1).ToArray());

            const string collectionsLinqProxy = ".First()";
            include = include.Replace(collectionsLinqProxy, "");

            return query.Include(include);
        }

        internal static string After(this string original, string search)
        {
            if (string.IsNullOrEmpty(original))
                return string.Empty;

            int index = original.IndexOf(search);
            return original.Substring(index + search.Length);
        }
    }
}

In Conery's MVC Storefront he created another layer called the "Service" layer which received IQueryable results from the respository and was responsible for applying various filters.

In all cases nobody should be interacting with the repository directly except the services layer.

Most flexible thing is to let Services interact with Repository whatever way they want, same as in above code (yet through one single point -as in example also- to write DRY code and find a place for optimization).
However, the more right way in terms of common DDD patterns is to use the "Specification" pattern, where you encapsulate all your filters, etc in Variables (Class Members, in LINQ typically of delegate types). LINQ can take big optimization benefit out of this when you combine it with "Compiled queries". If you google the {Specification Pattern} and {LINQ Compiled Queries} you'll get closer to what I mean here.

Answer 3

Having struggled to find a viable solution to this problem myself, there's what seems to be a good solution in the Implementing the Repository and Unit of Work Patterns in an ASP.NET MVC Application (9 of 10) article.

public virtual IEnumerable<TEntity> Get(
    Expression<Func<TEntity, bool>> filter = null,
    Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
    string includeProperties = "") 
    {
        IQueryable<TEntity> query = dbSet;

        if (filter != null)
        {
            query = query.Where(filter);
        }

        foreach (var includeProperty in includeProperties.Split
            (new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
        {
            query = query.Include(includeProperty);
        }

        if (orderBy != null)
        {
            return orderBy(query).ToList();
        }
        else
        {
            return query.ToList();
        }
    }

The article doesn't talk about this exact issue but does talk about generic, reusable repository methods.

So far this is all I've been able to come up with as a solution.

Answer 4

Rob's method really doesn't solve your core problem, and that is not wanting to write individual methods for each type of query you'd like to run, and unfortunately if you're not using IQueryable then that is what you are left with.

Sure the methods might be in the "service" layer, but it still means having to write "GetProductsByName, GetProductsByDate"...

The other method is something like:

GetProducts(QueryObject);

This might give you some benefit over using IQueryable in that you can constrain what is returned.

Answer 5

Exposing an IQueryable is a very viable solution and this is how most of the Repository implementations out there doing right now. (Including SharpArchitecture and FubuMVC contrib, as well.)

This is where you are wrong:

However, if you aren't using IQueryable then methods like GetAll() aren't really practical since lazy evaluation won't be taking place down the line. I don't want to return 10,000 records only to use 10 of them later.

This is not realy true. Your example is correct and you should rename GetAll() to a more informative name.

It DOESN'T return all of the items if you call it. That is what IQueryable is for. The concept is called "deferred loading", as it only loads the data (and makes database requests) when you enumerate the IQueryable.

So, let's say I have a method like this:

IQueryable<T> Retrieve() { ... }

Then, I can call it like this:

Repository.Retrieve<Customer>().Single(c => c.ID == myID);

This ONLY retrieves one row from the database.

And this:

Repository.Retrieve<Customer>().Where(c => c.FirstName == "Joe").OrderBy(c => c.LastName);

This also generates a corresponding query and is only executed when you enumerate it. (It generates an expression tree from the query, and then the query provider should translate that into an appropriate query against the data source.)

You can read more about it in this MSDN article.