Multi-Mapper to create object hierarchy

谁说胖子不能爱 提交于 2019-11-26 13:58:53
Sam Saffron

You are doing nothing wrong, it is just not the way the API was designed. All the Query APIs will always return an object per database row.

So, this works well on the many -> one direction, but less well for the one -> many multi-map.

There are 2 issues here:

  1. If we introduce a built-in mapper that works with your query, we would be expected to "discard" duplicate data. (Contacts.* is duplicated in your query)

  2. If we design it to work with a one -> many pair, we will need some sort of identity map. Which adds complexity.


Take for example this query which is efficient if you just need to pull a limited number of records, if you push this up to a million stuff get trickier, cause you need to stream and can not load everything into memory:

var sql = "set nocount on
DECLARE @t TABLE(ContactID int,  ContactName nvarchar(100))
INSERT @t
SELECT *
FROM Contacts
WHERE clientid=1
set nocount off 
SELECT * FROM @t 
SELECT * FROM Phone where ContactId in (select t.ContactId from @t t)"

What you could do is extend the GridReader to allow for the remapping:

var mapped = cnn.QueryMultiple(sql)
   .Map<Contact,Phone, int>
    (
       contact => contact.ContactID, 
       phone => phone.ContactID,
       (contact, phones) => { contact.Phones = phones };  
    );

Assuming you extend your GridReader and with a mapper:

public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
    (
    this GridReader reader,
    Func<TFirst, TKey> firstKey, 
    Func<TSecond, TKey> secondKey, 
    Action<TFirst, IEnumerable<TSecond>> addChildren
    )
{
    var first = reader.Read<TFirst>().ToList();
    var childMap = reader
        .Read<TSecond>()
        .GroupBy(s => secondKey(s))
        .ToDictionary(g => g.Key, g => g.AsEnumerable());

    foreach (var item in first)
    {
        IEnumerable<TSecond> children;
        if(childMap.TryGetValue(firstKey(item), out children))
        {
            addChildren(item,children);
        }
    }

    return first;
}

Since this is a bit tricky and complex, with caveats. I am not leaning towards including this in core.

FYI - I got Sam's answer working by doing the following:

First, I added a class file called "Extensions.cs". I had to change the "this" keyword to "reader" in two places:

using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;

namespace TestMySQL.Helpers
{
    public static class Extensions
    {
        public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
            (
            this Dapper.SqlMapper.GridReader reader,
            Func<TFirst, TKey> firstKey,
            Func<TSecond, TKey> secondKey,
            Action<TFirst, IEnumerable<TSecond>> addChildren
            )
        {
            var first = reader.Read<TFirst>().ToList();
            var childMap = reader
                .Read<TSecond>()
                .GroupBy(s => secondKey(s))
                .ToDictionary(g => g.Key, g => g.AsEnumerable());

            foreach (var item in first)
            {
                IEnumerable<TSecond> children;
                if (childMap.TryGetValue(firstKey(item), out children))
                {
                    addChildren(item, children);
                }
            }

            return first;
        }
    }
}

Second, I added the following method, modifying the last parameter:

public IEnumerable<Contact> GetContactsAndPhoneNumbers()
{
    var sql = @"
SELECT * FROM Contacts WHERE clientid=1
SELECT * FROM Phone where ContactId in (select ContactId FROM Contacts WHERE clientid=1)";

    using (var connection = GetOpenConnection())
    {
        var mapped = connection.QueryMultiple(sql)    
            .Map<Contact,Phone, int>     (        
            contact => contact.ContactID,        
            phone => phone.ContactID,
            (contact, phones) => { contact.Phones = phones; }      
        ); 
        return mapped;
    }
}

Check out https://www.tritac.com/blog/dappernet-by-example/ You could do something like this:

public class Shop {
  public int? Id {get;set;}
  public string Name {get;set;}
  public string Url {get;set;}
  public IList<Account> Accounts {get;set;}
}

public class Account {
  public int? Id {get;set;}
  public string Name {get;set;}
  public string Address {get;set;}
  public string Country {get;set;}
  public int ShopId {get;set;}
}

var lookup = new Dictionary<int, Shop>()
conn.Query<Shop, Account, Shop>(@"
                  SELECT s.*, a.*
                  FROM Shop s
                  INNER JOIN Account a ON s.ShopId = a.ShopId                    
                  ", (s, a) => {
                       Shop shop;
                       if (!lookup.TryGetValue(s.Id, out shop)) {
                           lookup.Add(s.Id, shop = s);
                       }
                       shop.Accounts.Add(a);
                       return shop;
                   },
                   ).AsQueryable();
var resultList = lookup.Values;

I got this from the dapper.net tests: https://code.google.com/p/dapper-dot-net/source/browse/Tests/Tests.cs#1343

Multi result set support

In your case it would be much better (and easier as well) to have a multi resultset query. This simply means that you should write two select statements:

  1. One that returns contacts
  2. And one that returns their phone numbers

This way your objects would be unique and wouldn't duplicate.

Clay

Here's a reusable solution that is pretty easy to use. It is a slight modification of Andrews answer.

public static IEnumerable<TParent> QueryParentChild<TParent, TChild, TParentKey>(
    this IDbConnection connection,
    string sql,
    Func<TParent, TParentKey> parentKeySelector,
    Func<TParent, IList<TChild>> childSelector,
    dynamic param = null, IDbTransaction transaction = null, bool buffered = true, string splitOn = "Id", int? commandTimeout = null, CommandType? commandType = null)
{
    Dictionary<TParentKey, TParent> cache = new Dictionary<TParentKey, TParent>();

    connection.Query<TParent, TChild, TParent>(
        sql,
        (parent, child) =>
            {
                if (!cache.ContainsKey(parentKeySelector(parent)))
                {
                    cache.Add(parentKeySelector(parent), parent);
                }

                TParent cachedParent = cache[parentKeySelector(parent)];
                IList<TChild> children = childSelector(cachedParent);
                children.Add(child);
                return cachedParent;
            },
        param as object, transaction, buffered, splitOn, commandTimeout, commandType);

    return cache.Values;
}

Example usage

public class Contact
{
    public int ContactID { get; set; }
    public string ContactName { get; set; }
    public List<Phone> Phones { get; set; } // must be IList

    public Contact()
    {
        this.Phones = new List<Phone>(); // POCO is responsible for instantiating child list
    }
}

public class Phone
{
    public int PhoneID { get; set; }
    public int ContactID { get; set; } // foreign key
    public string Number { get; set; }
    public string Type { get; set; }
    public bool IsActive { get; set; }
}

conn.QueryParentChild<Contact, Phone, int>(
    "SELECT * FROM Contact LEFT OUTER JOIN Phone ON Contact.ContactID = Phone.ContactID",
    contact => contact.ContactID,
    contact => contact.Phones,
    splitOn: "PhoneId");
shlgug

Based on Sam Saffron's (and Mike Gleason's) approach, here is a solution which will allow for multiple children and multiple levels.

using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;

namespace TestMySQL.Helpers
{
    public static class Extensions
    {
        public static IEnumerable<TFirst> MapChild<TFirst, TSecond, TKey>
            (
            this SqlMapper.GridReader reader,
            List<TFirst> parent,
            List<TSecond> child,
            Func<TFirst, TKey> firstKey,
            Func<TSecond, TKey> secondKey,
            Action<TFirst, IEnumerable<TSecond>> addChildren
            )
        {
            var childMap = child
                .GroupBy(secondKey)
                .ToDictionary(g => g.Key, g => g.AsEnumerable());
            foreach (var item in parent)
            {
                IEnumerable<TSecond> children;
                if (childMap.TryGetValue(firstKey(item), out children))
                {
                    addChildren(item, children);
                }
            }
            return parent;
        }
    }
}

Then you can have it read outside of the function.

using (var multi = conn.QueryMultiple(sql))
{
    var contactList = multi.Read<Contact>().ToList();
    var phoneList = multi.Read<Phone>().ToList;
    contactList = multi.MapChild
        (
            contactList,
            phoneList,
            contact => contact.Id, 
            phone => phone.ContactId,
            (contact, phone) => {contact.Phone = phone;}
        ).ToList();
    return contactList;
}

The map function can then be called again for the next child object using the same parent object. You can also implement splits on the parent or child read statements independently of the map function.

Here is a 'single to N' additional extension method

    public static TFirst MapChildren<TFirst, TSecond, TKey>
        (
        this SqlMapper.GridReader reader,
        TFirst parent,
        IEnumerable<TSecond> children,
        Func<TFirst, TKey> firstKey,
        Func<TSecond, TKey> secondKey,
        Action<TFirst, IEnumerable<TSecond>> addChildren
        )
    {
        if (parent == null || children == null || !children.Any())
        {
            return parent;
        }

        Dictionary<TKey, IEnumerable<TSecond>> childMap = children
            .GroupBy(secondKey)
            .ToDictionary(g => g.Key, g => g.AsEnumerable());

        if (childMap.TryGetValue(firstKey(parent), out IEnumerable<TSecond> foundChildren))
        {
            addChildren(parent, foundChildren);
        }

        return parent;
    }

I wanted to share my solution to this issue and see if anyone has any constructive feedback on the approach I've used?

I have a few requirements in the project I'm working on which I need to explain first up:

  1. I have to keep my POCO's as clean as possible as these classes will be publicly shared in an API wrapper.
  2. My POCO's are in a seperate Class Library because of the above requirement
  3. There are going to be multiple object hierarchy levels that will vary depending on data (so I cannot use a Generic Type Mapper or I'd have to write tons of them to cater for all possible eventualities)

So, what I have done is to get SQL to handle the 2nd - nth Level heirarchy by returning a Single JSON string as a column on the original row as follows (stripped out the other columns / properties etc to illustrate):

Id  AttributeJson
4   [{Id:1,Name:"ATT-NAME",Value:"ATT-VALUE-1"}]

Then, my POCO's are built up like the below :

public abstract class BaseEntity
{
    [KeyAttribute]
    public int Id { get; set; }
}

public class Client : BaseEntity
{
    public List<ClientAttribute> Attributes{ get; set; }
}
public class ClientAttribute : BaseEntity
{
    public string Name { get; set; }
    public string Value { get; set; }
}

Where the POCO's inherit from BaseEntity. (To illustrate I've chosen a fairly simple, single level heirarchy as shown by the "Attributes" property of the client object. )

I then have in my Data Layer the following "Data Class" which inherits from the POCO Client.

internal class dataClient : Client
{
    public string AttributeJson
    {
        set
        {
            Attributes = value.FromJson<List<ClientAttribute>>();
        }
    }
}

As you can see above, whats happening is that SQL is returning a column called "AttributeJson" which is mapped to the property AttributeJson in the dataClient class. This has only a setter which deserialises the JSON to the Attributes property on the inherited Client class. The dataClient Class is internal to the Data Access Layer and the ClientProvider (my data factory) returns the original Client POCO to the calling App / Library like so :

var clients = _conn.Get<dataClient>();
return clients.OfType<Client>().ToList();

Note that I'm using Dapper.Contrib and have added a new Get<T> Method that returns an IEnumerable<T>

There are a couple things to note with this solution:

  1. There's an obvious performance trade off with the JSON serialisation - I've benchmarked this against 1050 rows with 2 sub List<T> properties, each with 2 entities in the list and it clocks in at 279ms - which is acceptable for my projects needs - this is also with ZERO optimisation on the SQL side of things so I should be able to shave a few ms there.

  2. It does mean additional SQL queries are required to build up the JSON for each required List<T> property, but again, this suits me as I know SQL pretty well and am not so fluent on dynamics / reflection etc.. so this way I feel like I have more control over things as I actually understand whats happening under the hood :-)

There may well be a better solution than this one and if there is I would really appreciate hearing your thoughts - this is just the solution I came up with that so far fits my needs for this project (although this is experimental at the stage of posting).

Once we decided to move our DataAccessLayer to stored procedures, and these procedures are often return multiple linked results (example below).

Well, my approach is almost the same, but maybe little bit more comfortable.

This is how your code may look like:

using ( var conn = GetConn() )
{
    var res = await conn
        .StoredProc<Person>( procName, procParams )
        .Include<Book>( ( p, b ) => p.Books = b.Where( x => x.PersonId == p.Id ).ToList() )
        .Include<Course>( ( p, c ) => p.Courses = c.Where( x => x.PersonId == p.Id ).ToList() )
        .Include<Course, Mark>( ( c, m ) => c.Marks = m.Where( x => x.CourseId == c.Id ).ToList() )
        .Execute();
}


Let's break it down...

Extension:

public static class SqlExtensions
{
    public static StoredProcMapper<T> StoredProc<T>( this SqlConnection conn, string procName, object procParams )
    {
        return StoredProcMapper<T>
            .Create( conn )
            .Call( procName, procParams );
    }
}

Mapper:

public class StoredProcMapper<T>
{
    public static StoredProcMapper<T> Create( SqlConnection conn )
    {
        return new StoredProcMapper<T>( conn );
    }

    private List<MergeInfo> _merges = new List<MergeInfo>();

    public SqlConnection Connection { get; }
    public string ProcName { get; private set; }
    public object Parameters { get; private set; }

    private StoredProcMapper( SqlConnection conn )
    {
        Connection = conn;
        _merges.Add( new MergeInfo( typeof( T ) ) );
    }

    public StoredProcMapper<T> Call( object procName, object parameters )
    {
        ProcName = procName.ToString();
        Parameters = parameters;

        return this;
    }

    public StoredProcMapper<T> Include<TChild>( MergeDelegate<T, TChild> mapper )
    {
        return Include<T, TChild>( mapper );
    }

    public StoredProcMapper<T> Include<TParent, TChild>( MergeDelegate<TParent, TChild> mapper )
    {
        _merges.Add( new MergeInfo<TParent, TChild>( mapper ) );
        return this;
    }

    public async Task<List<T>> Execute()
    {
        if ( string.IsNullOrEmpty( ProcName ) )
            throw new Exception( $"Procedure name not specified! Please use '{nameof(Call)}' method before '{nameof( Execute )}'" );

        var gridReader = await Connection.QueryMultipleAsync( 
            ProcName, Parameters, commandType: CommandType.StoredProcedure );

        foreach ( var merge in _merges )
        {
            merge.Result = gridReader
                .Read( merge.Type )
                .ToList();
        }

        foreach ( var merge in _merges )
        {
            if ( merge.ParentType == null )
                continue;

            var parentMerge = _merges.FirstOrDefault( x => x.Type == merge.ParentType );

            if ( parentMerge == null )
                throw new Exception( $"Wrong parent type '{merge.ParentType.FullName}' for type '{merge.Type.FullName}'." );

            foreach ( var parent in parentMerge.Result )
            {
                merge.Merge( parent, merge.Result );
            }
        }

        return _merges
            .First()
            .Result
            .Cast<T>()
            .ToList();
    }

    private class MergeInfo
    {
        public Type Type { get; }
        public Type ParentType { get; }
        public IEnumerable Result { get; set; }

        public MergeInfo( Type type, Type parentType = null )
        {
            Type = type;
            ParentType = parentType;
        }

        public void Merge( object parent, IEnumerable children )
        {
            MergeInternal( parent, children );
        }

        public virtual void MergeInternal( object parent, IEnumerable children )
        {

        }
    }

    private class MergeInfo<TParent, TChild> : MergeInfo
    {
        public MergeDelegate<TParent, TChild> Action { get; }

        public MergeInfo( MergeDelegate<TParent, TChild> mergeAction )
            : base( typeof( TChild ), typeof( TParent ) )
        {
            Action = mergeAction;
        }

        public override void MergeInternal( object parent, IEnumerable children )
        {
            Action( (TParent)parent, children.Cast<TChild>() );
        }
    }

    public delegate void MergeDelegate<TParent, TChild>( TParent parent, IEnumerable<TChild> children );
}

That's all, but if you wanna do quick test, here is models and procedure for you:

Models:

public class Person
{
    public Guid Id { get; set; }
    public string Name { get; set; }

    public List<Course> Courses { get; set; }
    public List<Book> Books { get; set; }

    public override string ToString() => Name;
}

public class Book
{
    public Guid Id { get; set; }
    public Guid PersonId { get; set; }
    public string Name { get; set; }

    public override string ToString() => Name;
}

public class Course
{
    public Guid Id { get; set; }
    public Guid PersonId { get; set; }
    public string Name { get; set; }

    public List<Mark> Marks { get; set; }

    public override string ToString() => Name;
}

public class Mark
{
    public Guid Id { get; set; }
    public Guid CourseId { get; set; }
    public int Value { get; set; }

    public override string ToString() => Value.ToString();
}

SP:

if exists ( 
    select * 
    from sysobjects 
    where  
        id = object_id(N'dbo.MultiTest')
        and ObjectProperty( id, N'IsProcedure' ) = 1 )
begin
    drop procedure dbo.MultiTest
end
go

create procedure dbo.MultiTest
    @PersonId UniqueIdentifier
as
begin

    declare @tmpPersons table 
    (
        Id UniqueIdentifier,
        Name nvarchar(50)
    );

    declare @tmpBooks table 
    (
        Id UniqueIdentifier,
        PersonId UniqueIdentifier,
        Name nvarchar(50)
    )

    declare @tmpCourses table 
    (
        Id UniqueIdentifier,
        PersonId UniqueIdentifier,
        Name nvarchar(50)
    )

    declare @tmpMarks table 
    (
        Id UniqueIdentifier,
        CourseId UniqueIdentifier,
        Value int
    )

--------------------------------------------------

    insert into @tmpPersons
    values
        ( '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Иван' ),
        ( '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Василий' ),
        ( '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Алефтина' )


    insert into @tmpBooks
    values
        ( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Математика' ),
        ( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Физика' ),
        ( NewId(), '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Книга Геометрия' ),

        ( NewId(), '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Книга Биология' ),
        ( NewId(), '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Книга Химия' ),

        ( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга История' ),
        ( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга Литература' ),
        ( NewId(), '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Книга Древне-шумерский диалект иврита' )


    insert into @tmpCourses
    values
        ( '30945b68-a6ef-4da8-9a35-d3b2845e7de3', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Математика' ),
        ( '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Физика' ),
        ( '92bbefd1-9fec-4dc7-bb58-986eadb105c8', '576fb8e8-41a2-43a9-8e77-a8213aa6e387', N'Геометрия' ),

        ( '923a2f0c-c5c7-4394-847c-c5028fe14711', '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Биология' ),
        ( 'ace50388-eb05-4c46-82a9-5836cf0c988c', '467953a5-cb5f-4d06-9fad-505b3bba2058', N'Химия' ),

        ( '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'История' ),
        ( '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Литература' ),
        ( '73ac366d-c7c2-4480-9513-28c17967db1a', '52a719bf-6f1f-48ac-9e1f-4532cfc70d96', N'Древне-шумерский диалект иврита' )

    insert into @tmpMarks
    values
        ( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 98 ),
        ( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 87 ),
        ( NewId(), '30945b68-a6ef-4da8-9a35-d3b2845e7de3', 76 ),

        ( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 89 ),
        ( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 78 ),
        ( NewId(), '7881f090-ccd6-4fb9-a1e0-ff4ff5c18450', 67 ),

        ( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 79 ),
        ( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 68 ),
        ( NewId(), '92bbefd1-9fec-4dc7-bb58-986eadb105c8', 75 ),
        ----------
        ( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 198 ),
        ( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 187 ),
        ( NewId(), '923a2f0c-c5c7-4394-847c-c5028fe14711', 176 ),

        ( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 189 ),
        ( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 178 ),
        ( NewId(), 'ace50388-eb05-4c46-82a9-5836cf0c988c', 167 ),
        ----------
        ( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 8 ),
        ( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 7 ),
        ( NewId(), '53ea69fb-6cc4-4a6f-82c2-0afbaa8cb410', 6 ),

        ( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 9 ),
        ( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 8 ),
        ( NewId(), '7290c5f7-1000-4f44-a5f0-6a7cf8a8efab', 7 ),

        ( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 9 ),
        ( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 8 ),
        ( NewId(), '73ac366d-c7c2-4480-9513-28c17967db1a', 5 )

--------------------------------------------------

    select * from @tmpPersons
    select * from @tmpBooks
    select * from @tmpCourses
    select * from @tmpMarks

end
go
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!