Fastest way to map result of SqlDataReader to object

Question

I\'m comparing materialize time between Dapper and ADO.NET and Dapper. Ultimately, Dapper tend to faster than ADO.NET, though the first time a given fetch query was executed

Answer 1

When in doubt regarding anything db or reflection, I ask myself, "what would Marc Gravell do?".

In this case, he would use FastMember! And you should too. It's the underpinning to the data conversions in Dapper, and can easily be used to map your own DataReader to an object (should you not want to use Dapper).

Below is an extension method converting a SqlDataReader into something of type T:

PLEASE NOTE: This code implies a dependency on FastMember and is written for .NET Core (though could easily be converted to .NET Framework/Standard compliant code).

public static T ConvertToObject<T>(this SqlDataReader rd) where T : class, new()
{
    Type type = typeof(T);
    var accessor = TypeAccessor.Create(type);
    var members = accessor.GetMembers();
    var t = new T();

    for (int i = 0; i < rd.FieldCount; i++)
    {
        if (!rd.IsDBNull(i))
        {
            string fieldName = rd.GetName(i);

            if (members.Any(m => string.Equals(m.Name, fieldName, StringComparison.OrdinalIgnoreCase)))
            {
                accessor[t, fieldName] = rd.GetValue(i);
            }
        }
    }

    return t;
}

Answer 2

Took the method from pimbrouwers' answer and optimized it slightly. Reduce LINQ calls.

Maps only properties found in both the object and data field names. Handles DBNull. Other assumption made is your domain model properties absolutely equals table column/field names.

/// <summary>
/// Maps a SqlDataReader record to an object.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="dataReader"></param>
/// <param name="newObject"></param>
public static void MapDataToObject<T>(this SqlDataReader dataReader, T newObject)
{
    if (newObject == null) throw new ArgumentNullException(nameof(newObject));

    // Fast Member Usage
    var objectMemberAccessor = TypeAccessor.Create(newObject.GetType());
    var propertiesHashSet =
            objectMemberAccessor
            .GetMembers()
            .Select(mp => mp.Name)
            .ToHashSet();

    for (int i = 0; i < dataReader.FieldCount; i++)
    {
        if (propertiesHashSet.Contains(dataReader.GetName(i)))
        {
            objectMemberAccessor[newObject, dataReader.GetName(i)]
                = dataReader.IsDBNull(i) ? null : dataReader.GetValue(i);
        }
    }
}

Sample Usage:

public async Task<T> GetAsync<T>(string storedProcedureName, SqlParameter[] sqlParameters = null) where T : class, new()
{
    using (var conn = new SqlConnection(_connString))
    {
        var sqlCommand = await GetSqlCommandAsync(storedProcedureName, conn, sqlParameters);
        var dataReader = await sqlCommand.ExecuteReaderAsync(CommandBehavior.CloseConnection);

        if (dataReader.HasRows)
        {
            var newObject = new T();

            if (await dataReader.ReadAsync())
            { dataReader.MapDataToObject(newObject); }

            return newObject;
        }
        else
        { return null; }
    }
}

Answer 3

Perhaps the approach I will present isn't the most efficient but gets the job done with very little coding effort. The main benefit I see here is that you don't have to deal with data structure other than building a compatible (mapable) object.

If you convert the SqlDataReader to DataTable then serialize it using JsonConvert.SerializeObject you can then deserialize it to a known object type using JsonConvert.DeserializeObject

Here is an example of implementation:

        SqlDataReader reader = null;
        SqlConnection myConnection = new SqlConnection();
        myConnection.ConnectionString = ConfigurationManager.ConnectionStrings["DatabaseConnection"].ConnectionString;
        SqlCommand sqlCmd = new SqlCommand();
        sqlCmd.CommandType = CommandType.Text;
        sqlCmd.CommandText = "SELECT * FROM MyTable";
        sqlCmd.Connection = myConnection;
        myConnection.Open();
        reader = sqlCmd.ExecuteReader();

        var dataTable = new DataTable();
        dataTable.Load(reader);

        List<MyObject> myObjects = new List<MyObject>();

        if (dataTable.Rows.Count > 0)
        {
            var serializedMyObjects = JsonConvert.SerializeObject(dataTable);
            // Here you get the object
            myObjects = (List<MyObject>)JsonConvert.DeserializeObject(serializedMyObjects, typeof(List<MyObject>));
        }

        myConnection.Close();

Answer 4

I took both pimbrouwers and HouseCat's answers and come up with me. In my scenario, the column name in database has snake case format.

public static T ConvertToObject<T>(string query) where T : class, new()
    {
        using (var conn = new SqlConnection(AutoConfig.ConnectionString))
        {
            conn.Open();
            var cmd = new SqlCommand(query) {Connection = conn};
            var rd = cmd.ExecuteReader();
            var mappedObject = new T();

            if (!rd.HasRows) return mappedObject;
            var accessor = TypeAccessor.Create(typeof(T));
            var members = accessor.GetMembers();
            if (!rd.Read()) return mappedObject;
            for (var i = 0; i < rd.FieldCount; i++)
            {
                var columnNameFromDataTable = rd.GetName(i);
                var columnValueFromDataTable = rd.GetValue(i);

                var splits = columnNameFromDataTable.Split('_');
                var columnName = new StringBuilder("");
                foreach (var split in splits)
                {
                    columnName.Append(CultureInfo.InvariantCulture.TextInfo.ToTitleCase(split.ToLower()));
                }

                var mappedColumnName = members.FirstOrDefault(x =>
                    string.Equals(x.Name, columnName.ToString(), StringComparison.OrdinalIgnoreCase));

                if(mappedColumnName == null) continue;
                var columnType = mappedColumnName.Type;

                if (columnValueFromDataTable != DBNull.Value)
                {
                    accessor[mappedObject, columnName.ToString()] = Convert.ChangeType(columnValueFromDataTable, columnType);
                }
            }

            return mappedObject;
        }
    }

Answer 5

This is based on the other answers but I used standard reflection to read the properties of the class you want to instantiate and fill it from the dataReader. You could also store the properties using a dictionary persisted b/w reads.

Initialize a dictionary containing the properties from the type with their names as the keys.

var type = typeof(Foo);
var properties = type.GetProperties(BindingFlags.Public | BindingFlags.Instance);
var propertyDictionary = new Dictionary<string,PropertyInfo>();
foreach(var property in properties)
{
    if (!property.CanWrite) continue;
    propertyDictionary.Add(property.Name, property);
}

The method to set a new instance of the type from the DataReader would be like:

var foo = new Foo();
//retrieve the propertyDictionary for the type
for (var i = 0; i < dataReader.FieldCount; i++)
{
    var n = dataReader.GetName(i);
    PropertyInfo prop;
    if (!propertyDictionary.TryGetValue(n, out prop)) continue;
    var val = dataReader.IsDBNull(i) ? null : dataReader.GetValue(i);
    prop.SetValue(foo, val, null);
}
return foo;

If you want to write an efficient generic class dealing with multiple types you could store each dictionary in a global dictionary>.

Answer 6

Modified @HouseCat's solution to be case insensitive:

    /// <summary>
    /// Maps a SqlDataReader record to an object. Ignoring case.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="dataReader"></param>
    /// <param name="newObject"></param>
    /// <remarks>https://stackoverflow.com/a/52918088</remarks>
    public static void MapDataToObject<T>(this SqlDataReader dataReader, T newObject)
    {
        if (newObject == null) throw new ArgumentNullException(nameof(newObject));

        // Fast Member Usage
        var objectMemberAccessor = TypeAccessor.Create(newObject.GetType());
        var propertiesHashSet =
                objectMemberAccessor
                .GetMembers()
                .Select(mp => mp.Name)
                .ToHashSet(StringComparer.InvariantCultureIgnoreCase);

        for (int i = 0; i < dataReader.FieldCount; i++)
        {
            var name = propertiesHashSet.FirstOrDefault(a => a.Equals(dataReader.GetName(i), StringComparison.InvariantCultureIgnoreCase));
            if (!String.IsNullOrEmpty(name))
            {
                objectMemberAccessor[newObject, name]
                    = dataReader.IsDBNull(i) ? null : dataReader.GetValue(i);
            }
        }
    }

EDIT: This does not work for List<T> or multiple tables in the results.

EDIT2: Changing the calling function to this works for lists. I am just going to return a list of objects no matter what and get the first index if I was expecting a single object. I haven't looked into multiple tables yet but I will.

    public async Task<List<T>> ExecuteReaderAsync<T>(string storedProcedureName, SqlParameter[] sqlParameters = null) where T : class, new()
    {
        var newListObject = new List<T>();
        using (var conn = new SqlConnection(_connectionString))
        {
            conn.Open();
            SqlCommand sqlCommand = GetSqlCommand(conn, storedProcedureName, sqlParameters);
            using (var dataReader = await sqlCommand.ExecuteReaderAsync(CommandBehavior.Default))
            {
                if (dataReader.HasRows)
                {
                    while (await dataReader.ReadAsync())
                    {
                        var newObject = new T();
                        dataReader.MapDataToObject(newObject);
                        newListObject.Add(newObject);
                    }
                }
            }
        }
        return newListObject;
    }