How To Count Associated Entities Without Fetching Them In Entity Framework

Question

I\'ve been wondering about this one for a while now, so I thought it would be worth using my first Stack Overflow post to ask about it.

Imagine I have a discussion with

Answer 1

I know this is an old question but it seems to be an ongoing problem and none of the answers above provide a good way to deal with SQL aggregates in list views.

I am assuming straight POCO models and Code First like in the templates and examples. While the SQL View solution is nice from a DBA point of view, it re-introduces the challenge of maintaining both code and database structures in parallel. For simple SQL aggregate queries, you won't see much speed gain from a View. What you really need to avoid are multiple (n+1) database queries, as in the examples above. If you have 5000 parent entities and you are counting child entities (e.g. messages per discussion), that's 5001 SQL queries.

You can return all those counts in a single SQL query. Here's how.

1. Add a placeholder property to your class model using the [NotMapped] data annotation from the System.ComponentModel.DataAnnotations.Schema namespace. This gives you a place to store the calculated data without actually adding a column to your database or projecting to unnecessary View Models.

   ...
   using System.ComponentModel.DataAnnotations;
   using System.ComponentModel.DataAnnotations.Schema;
   
   namespace MyProject.Models
   {
       public class Discussion
       {
           [Key]
           public int ID { get; set; }
   
           ...
   
           [NotMapped]
           public int MessageCount { get; set; }
   
           public virtual ICollection<Message> Messages { get; set; }
       }
   }
   

2. In your Controller, get the list of parent objects.

   var discussions = db.Discussions.ToList();
   

3. Capture the counts in a Dictionary. This generates a single SQL GROUP BY query with all parent IDs and child object counts. (Presuming DiscussionID is the FK in Messages.)

   var _counts = db.Messages.GroupBy(m => m.DiscussionID).ToDictionary(d => d.Key, d => d.Count());
   

4. Loop through the parent objects, look up the count from the dictionary, and store in the placeholder property.

   foreach (var d in discussions)
       {
           d.MessageCount = (_counts.ContainsKey(d.ID)) ? _counts[d.ID] : 0;
       }
   

5. Return your discussion list.

   return View(discussions);
   

6. Reference the MessageCount property in the View.

   @foreach (var item in Model) {
       ...
       @item.MessageCount
       ...
   }
   

Yes, you could just stuff that Dictionary into the ViewBag and do the lookup directly in the View, but that muddies your view with code that doesn't need to be there.

In the end, I wish EF had a way to do "lazy counting". The problem with both lazy and explicit loading is you're loading the objects. And if you have to load to count, that's a potential performance problem. Lazy counting wouldn't solve the n+1 problem in list views but it sure would be nice to be able to just call @item.Messages.Count from the View without having to worry about potentially loading tons of unwanted object data.

Hope this helps.

Answer 2

I've come across the same issue when dealing with multiple mappers including EF and DevExpress XPO (which doesn't even allow a single entity to map to multiple tables). What I found to be the best solution is to basically use the EDMX and T4 templates to generate updatable views in SQL Server (with instead of triggers) and that way you have low level control over the sql so you can do sub-queries in select clause, use all kinds of complex joins to bring in data and so on.

Answer 3

I don't have a direct answer but can only point you to the following comparison between NHibernate and EF 4.0 which seems to suggest that even in EF 4.0 there is no out of the box support for getting counts of a related entity collection without retrieving the collection.

http://ayende.com/Blog/archive/2010/01/05/nhibernate-vs.-entity-framework-4.0.aspx

I've upvoted and starred your question. Hopefully someone will chime in with a workable solution.

Answer 4

If you are using Entity Framework 4.1 or later, you can use:

var discussion = _repository.GetDiscussionCategory(id);

// Count how many messages the discussion has 
var messageCount = context.Entry(discussion)
                      .Collection(d => d.Messages)
                      .Query()
                      .Count();

Source: http://msdn.microsoft.com/en-US/data/jj574232

Answer 5

If this isn't a one off, and you find yourself needing to count a number of different associated entities, a database view might be a simpler (and potentially more appropriate) choice:

1. Create your database view.

   Assuming you want all of the original entity properties plus the associated message count:

   CREATE VIEW DiscussionCategoryWithStats AS
   SELECT dc.*,
         (SELECT count(1) FROM Messages m WHERE m.DiscussionCategoryId = dc.Id)
             AS MessageCount
   FROM DiscussionCategory dc
   

   (If you're using Entity Framework Code First Migrations, see this SO answer on how to create a view.)

2. In EF, simply use the view instead of the original entity:

   // You'll need to implement this!
   DiscussionCategoryWithStats dcs = _repository.GetDiscussionCategoryWithStats(id);
   
   int i = dcs.MessageCount;
   ...
   

Answer 6

Easy; just project onto a POCO (or anonymous) type:

var q = from d in Model.Discussions
        select new DiscussionPresentation
        {
            Subject = d.Subject,
            MessageCount = d.Messages.Count(),
        };

When you look at the generated SQL, you'll see that the Count() is done by the DB server.

Note that this works in both EF 1 and EF 4.