How to add CsvHelper records to DataTable to use for SqlBulkCopy to the database

Question

I am trying to read a CSV file with CsvHelper, load each record into a DataTable, and then use SqlBulkCopy to insert the data into a database table. With the current code, I get

Answer 1

I liked the answer from @JoshClose, but I found while( csv.Read() ) to be considerably slower than csv.GetRecords<{Class}>().ToList(). It also doesn't correctly handle many nullable types like int? when the value returned should be DBNull. My answer is to have CsvHelper import a list of dynamic records and then use a couple helper methods to auto map to the DataTable.

var records = csv.GetRecords().ToList();

foreach ( record in records )
{
    var row = dt.NewRow();

    var recordDictionary = DynamicToDictionary( record );

    foreach( DataColumn column in dt.Columns )
    {
        row[column.ColumnName] = GetColumnValue( column, recordDictionary );
    }

    dt.Rows.Add( row );
}

The DynamicToDictionary method handles case sensitivity and header white space. I convert the dynamic object to a Dictionary object that ignores case sensitivity and removes header white space. This could be skipped and the dynamic object passed directly to GetColumnValue if this isn't an issue.

public Dictionary DynamicToDictionary(dynamic dynObj)
{
    var dictionary = new Dictionary(StringComparer.OrdinalIgnoreCase);

    foreach (var kvp in (IDictionary) dynObj)
    {
        var obj = kvp.Value;

        // Remove white space.
        var name = new string(kvp.Key.ToCharArray().Where(c => !char.IsWhiteSpace(c)).ToArray());

        dictionary.Add(name, obj);
    }

    return dictionary;
}

The GetColumnValue method finds and converts the dynamic record value into the proper DataTable column value.

public object GetColumnValue(DataColumn column, IDictionary dynamicDictionary)
{
    object value;

    // Return DBNull if the column name isn't found.
    if (!dynamicDictionary.TryGetValue(column.ColumnName, out value))
    {
        return DBNull.Value;
    }

    // Null values come in as empty strings.
    if (column.AllowDBNull && column.DataType != typeof(string) && (string)value == "")
    {
        return DBNull.Value;
    }

    if (column.DataType == typeof(bool))
    {
        return (string)value != "0" && ((string)value).ToLower() != "false";
    }

    return value;
}