Checking for Nulls on DB Record Mapping

Question

How can I check for db null values in the attached code? Please understand I am a new C# convert...

What this code does is takes a IDataReader object and converts and maps it to a strongly-typed list of objects. But what I am finding is it completely errors out when there are null columns returned in the reader.

Converter

internal class Converter<T> where T : new()
{
    // Declare our _converter delegate
    readonly Func<IDataReader, T> _converter;
    // Declare our internal dataReader
    readonly IDataReader dataReader;

    // Build our mapping based on the properties in the class/type we've passed in to the class
    private Func<IDataReader, T> GetMapFunc()
    {
        // declare our field count
        int _fc = dataReader.FieldCount;
        // declare our expression list
        List<Expression> exps = new List<Expression>();
        // build our parameters for the expression tree
        ParameterExpression paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
        ParameterExpression targetExp = Expression.Variable(typeof(T));
        // Add our expression tree assignment to the exp list
        exps.Add(Expression.Assign(targetExp, Expression.New(targetExp.Type)));
        //does int based lookup
        PropertyInfo indexerInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(int) });
        // grab a collection of column names from our data reader
        var columnNames = Enumerable.Range(0, _fc).Select(i => new { i, name = dataReader.GetName(i) }).AsParallel();
        // loop through all our columns and map them properly
        foreach (var column in columnNames)
        {
            // grab our column property
            PropertyInfo property = targetExp.Type.GetProperty(column.name, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
            // check if it's null or not
            if (property != null)
            {
                // build our expression tree to map the column to the T
                ConstantExpression columnNameExp = Expression.Constant(column.i);
                IndexExpression propertyExp = Expression.MakeIndex(paramExp, indexerInfo, new[] { columnNameExp });
                UnaryExpression convertExp = Expression.Convert(propertyExp, property.PropertyType);
                BinaryExpression bindExp = Expression.Assign(Expression.Property(targetExp, property), convertExp);
                // add it to our expression list
                exps.Add(bindExp);
            }
        }
        // add the originating map to our expression list
        exps.Add(targetExp);
        // return a compiled cached map
        return Expression.Lambda<Func<IDataReader, T>>(Expression.Block(new[] { targetExp }, exps), paramExp).Compile();
    }

    // initialize
    internal Converter(IDataReader dataReader)
    {
        // initialize the internal datareader
        this.dataReader = dataReader;
        // build our map
        _converter = GetMapFunc();
    }

    // create and map each column to it's respective object
    internal T CreateItemFromRow()
    {
        return _converter(dataReader);
    }
}

Mapper

    private static IList<T> Map<T>(DbDataReader dr) where T : new()
    {
        try
        {
            // initialize our returnable list
            List<T> list = new List<T>();
            // fire up the lamda mapping
            var converter = new Converter<T>(dr);
            while (dr.Read())
            {
                // read in each row, and properly map it to our T object
                var obj = converter.CreateItemFromRow();
                // add it to our list
                list.Add(obj);
            }
            // reutrn it
            return list;
        }
        catch (Exception ex)
        {
            // make sure this method returns a default List
            return default(List<T>);
        }
    }

I just don't quite understand where the column to typed object happens in here, so I'd try to do it myself... but I just don;t know where it is.

I know this probably won't help much, but the error I am getting is:

Unable to cast object of type 'System.DBNull' to type 'System.String'.

and it happens on the

internal T CreateItemFromRow()
    {
        return _converter(dataReader); //<-- Here
    }

Note

This does not happen if I wrap the columns in the query itself with an ISNULL(column, ''), but I am sure you can understand that this is surely not a solution

Answer 1

The problem lies in the line convertExp = Expression.Convert(propertyExp, property.PropertyType). You can't expect to convert DbNull value to its equivalent in framework type. This is especially nasty when your type is a value type. One option is to check if the read value from db is DbNull.Value and in case yes, you need to find a compatible value yourself. In some cases people are ok with default values of those types in C#. If you have to do this

property = value == DBNull.Value ? default(T): value;

a generic implementation would look like (as far as the foreach in your converter class goes):

foreach (var column in columns)
{
    var property = targetExp.Type.GetProperty(
        column.name,
        BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
    if (property == null)
        continue;

    var columnIndexExp = Expression.Constant(column.i);
    var propertyExp = Expression.MakeIndex(
        paramExp, indexerInfo, new[] { columnIndexExp });
    var convertExp = Expression.Condition(
        Expression.Equal(
            propertyExp, 
            Expression.Constant(DBNull.Value)), 
        Expression.Default(property.PropertyType), 
        Expression.Convert(propertyExp, property.PropertyType));
    var bindExp = Expression.Assign(
        Expression.Property(targetExp, property), convertExp);
    exps.Add(bindExp);
}

Now this does an equivalent of

property = reader[index] == DBNull.Value ? default(T): reader[index];

You could avoid the double lookup of the reader by assigning it to a variable and using its value in the conditional check. So this should be marginally better, but a lil' more complex:

foreach (var column in columns)
{
    var property = targetExp.Type.GetProperty(
        column.name,
        BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
    if (property == null)
        continue;

    var columnIndexExp = Expression.Constant(column.i);
    var cellExp = Expression.MakeIndex(
        paramExp, indexerInfo, new[] { columnIndexExp });
    var cellValueExp = Expression.Variable(typeof(object), "o7thPropValue");
    var convertExp = Expression.Condition(
        Expression.Equal(
            cellValueExp, 
            Expression.Constant(DBNull.Value)), 
        Expression.Default(property.PropertyType), 
        Expression.Convert(cellValueExp, property.PropertyType));
    var cellValueReadExp = Expression.Block(new[] { cellValueExp },
        Expression.Assign(cellValueExp, cellExp), convertExp);
    var bindExp = Expression.Assign(
        Expression.Property(targetExp, property), cellValueReadExp);
    exps.Add(bindExp);
}

This does the conditional check this way:

value = reader[index];
property = value == DBNull.Value ? default(T): value;

Answer 2

This is one of the most annoying problems in dealing with datasets in general.

The way I normally get around it is to convert the DBNull value to something more useful, like an actual null or even a blank string in some cases. This can be done in a number of ways, but just recently I've taken to using extension methods.

public static T? GetValueOrNull<T>(this object value) where T : struct
        {
            return value == null || value == DBNull.Value ? (T?) null : (T) Convert.ChangeType(value, typeof (T));
        }

A handy extension method for nullable types, so for example:

int? myInt = DataSet.Tables[0].Rows[0]["DBNullInt"].GetValueOrNull<int>();

Or a more generic one to just convert a DBNull in to a null:

public static object GetValueOrNull(this object value)
        {
            return value == DBNull.Value ? null : value;
        }

string myString DataSet.Tables[0].Rows[0]["DBNullString"].GetValueOrNull();

You'll then get a null string, rather than trying to put a DBNull in to a string.

Hopefully that may help you a little.

Answer 3

As I come across this problem recently both

Expression.TypeIs(propertyExp,typeof(DBNull));

and

Expression.Equal(propertyExp,Expression.Constant(DBNull.Value));

didn't work for me as they did increase memory allocation (which is my primary concern in this case)

here is the benchmark for both mapper approach compare to Dapper on 10K rows query.

TypeIs

and Equal

so to fix this problem it came out that an IDataRecord is able to call "IsDBNull" to check whether the column in current reader is DBNull or not

and can be write as expression like

var isReaderDbNull = Expression.Call(paramExp, "IsDBNull", null, readerIndex);

finally, I end up with this solution

and now the performance is acceptable again.