Group and Select First From Two Tables Linq

Question

I\'m trying to create a simple query using EFCore, returning the list of people i\'m conversing with, and the last message that was sent between the two of us (pretty much like

Answer 1

You should be able to get your visitor information via association without explicitly trying to join/group on it.

Apologies for switching to Fluent syntax, I really dislike the Linq QL, it always seems so forced to do anything through it... :)

Your original query:

from c in ChatMessages
orderby c.CreatedAt descending 
group c by c.VisitorId  into x
select x.First()

or

var groupedMessages = context.ChatMessages
    .OrderByDescending(c => c.CreatedAt)
    .GroupBy(c => c.VisitorId)
    .First();

To group by Visitor:

var groupedMessages = context.ChatMessages
    .OrderByDescending(c => c.CreatedAt)
    .GroupBy(c => c.Visitor)
    .First();

This will give you a Key as a Visitor entity, with the messages for that visitor. However, this somewhat begs the question, why?

var visitorsWithMessages = context.Visitors.Include(v => v.VisitorChatMessages);

This loads the visitors and eager-loads their associated chat messages. Entities satisfy queries against the data relationships. For consumption though we care about details like ensuring the chat messages are ordered, or possibly filtered.

To project that into a suitable structure I'd use view models for the visitor and chat message to optimize the query to cover just the details I care about, and present them in the way I care about:

var visitorsWithMessages = context.Visitors
   // insert .Where() clause here to filter which Visitors we care to retrieve...
   .Select(v => new VisitorViewModel
   {
      Id = v.Id,
      Name = v.Name,
      RecentChatMessages = v.VisitorChatMessages
         .OrderByDescending(c => c.CreatedAt)
         .Select(c => new ChatMessageViewModel
         {
            Id = c.Id,
            Message = c.Message,
            CreatedAt = c.CreatedAt,
            CreatedBy = c.User.UserName ?? "Anonymous"
         }).Take(10).ToList()
    }).ToList();

This uses projection and EF's mapped relationships to get a list of Visitors and up to 10 of their most recent chat messages. I populate simple POCO view model classes which contain the fields I care about. These view models can be safely returned from methods or serialized to a view/API consumer without risking tripping up lazy loading. If I just need the data and don't need to send it anywhere, I can use anonymous types to get the fields I care about. We don't need to explicitly join entities together, you only need to do that for entities that deliberately do not have FK relationships mapped. We also do not need to deliberately eager-load entities either. The Select will compose an optimized SQL statement for the columns we need.

You can consume those results in the view and render based on Visitor + Messages or even dive deeper if you want to display a list of each visitors 10 most recent messages /w visitor details as a flattened list of messages:

Edit: The below query may have had an issue by accessing "v." after the .SelectMany. Corrected to "c.Visitor."

var recentMessages = context.Visitors
   .SelectMany(v => v.VisitorChatMessages
      .OrderByDescending(c => c.CreatedAt)
      .Select(c => new VisitorChatMessageViewModel
      {
          Id = c.Id,
          VisitorId = c.Visitor.Id,
          Message = c.Message,
          CreatedAt = c.CreatedAt,
          CreatedBy = c.User.UserName ?? "Anonymous",
          Visitor = c.Visitor.Name
      }).Take(10)
    }).ToList();

No idea how to do that in Linq QL though. :)

Edit: That last example will give you the last 10 messages with their applicable Visitor detail. (Name)

To get the last 100 messages for example and group them by their visitor:

var recentMessages = context.Visitors
   .SelectMany(v => v.VisitorChatMessages
      .OrderByDescending(c => c.CreatedAt)
      .Select(c => new VisitorChatMessageViewModel
      {
          Id = c.Id,
          VisitorId = c.Visitor.Id,
          Message = c.Message,
          CreatedAt = c.CreatedAt,
          CreatedBy = c.User.UserName ?? "Anonymous",
          Visitor = c.Visitor.Name // Visitor Name, or could be a ViewModel for more info about Visitor...
      }).Take(100)
    }).GroupBy(x => x.Visitor).ToList();

If you select a VisitorViewModel for Visitor instead of ".Visitor.Name" then your grouping key will have access to Name, Id, etc. whatever you select from the associated Visitor.

Answer 2

The query you are looking for standardly is expressed in LINQ to Entities (EF) with something like this (no joins, no GroupBy, use navigation properties):

var query = context.Visitors
    .Select(v => new
    {
        Visitor = v,
        Message = v.VisitorChatMessages
            .OrderByDescending(m => m.CreatedAt)
            .FirstOrDefault()
    });

But here is the trap. EF6 creates quite inefficient SQL query, and EF Core until now produces (again quite inefficient) N + 1 SQL queries.

But this is changing in EF Core 3.0 in a positive direction! Usually (and still) I don't recommend using the preview (beta) versions of EF Core 3.0, because they are rewriting the whole query translation/processing pipeline, so many things don't work as expected.

But today I've updated my EF Core test environment to EF Core 3.0 Preview 9 and I'm pleased to see that the above query now nicely translates to the following single SQL query:

  SELECT [v].[Id], [v].[CreatedAt], [v].[CreatedBy], [v].[Email], [v].[Fingerprint], [v].[IP], [v].[IsDeleted], [v].[LastUpdatedAt], [v].[LastUpdatedBy], [v].[Name], [v].[Phone], [t0].[Id], [t0].[CreatedAt], [t0].[CreatedBy], [t0].[IsDeleted], [t0].[IsFromVisitor], [t0].[LastUpdatedAt], [t0].[LastUpdatedBy], [t0].[Message], [t0].[UserId], [t0].[VisitorId]
  FROM [Visitors] AS [v]
  LEFT JOIN (
      SELECT [t].[Id], [t].[CreatedAt], [t].[CreatedBy], [t].[IsDeleted], [t].[IsFromVisitor], [t].[LastUpdatedAt], [t].[LastUpdatedBy], [t].[Message], [t].[UserId], [t].[VisitorId]
      FROM (
          SELECT [c].[Id], [c].[CreatedAt], [c].[CreatedBy], [c].[IsDeleted], [c].[IsFromVisitor], [c].[LastUpdatedAt], [c].[LastUpdatedBy], [c].[Message], [c].[UserId], [c].[VisitorId], ROW_NUMBER() OVER(PARTITION BY [c].[VisitorId] ORDER BY [c].[CreatedAt] DESC) AS [row]
          FROM [ChatMessages] AS [c]
      ) AS [t]
      WHERE [t].[row] <= 1
  ) AS [t0] ON [v].[Id] = [t0].[VisitorId]

Note the beautiful utilization of the ROW_NUMBER() OVER (PARTITION BY ORDER BY) construct. This is the first time EF query translation does that ever. I'm excited. Good job, EF Core team!

---

Update: The exact equivalent of your first query (which btw fails with runtime exception in Preview 9)

from c in context.ChatMessages
orderby c.CreatedAt descending 
group c by c.VisitorId  into x
select x.First()

but with additional information is

from v in context.Visitors
from c in v.VisitorChatMessages
    .OrderByDescending(c => c.CreatedAt)
    .Take(1)
orderby c.CreatedAt descending
select new
{
    Visitor = v,
    Message = c
})

The generated SQL is pretty much the same - just the LEFT OUTER JOIN becomes INNER JOIN and there is additional ORDER BY at the end.

Looks like that to make this work, it's essential to avoid GroupBy and use GroupJoin (which collection navigation property represents in LINQ to Entities queries) or correlated SelectMany to achieve the desired grouping.