Received an invalid column length from the bcp client for colid 6

Question

I want to bulk upload csv file data to sql server 2005 from c# code but I am encountering the below error -

Received an invalid column length from th

Answer 1

I faced a similar kind of issue while passing a string to Database table using SQL BulkCopy option. The string i was passing was of 3 characters whereas the destination column length was varchar(20). I tried trimming the string before inserting into DB using Trim() function to check if the issue was due to any space (leading and trailing) in the string. After trimming the string, it worked fine.

You can try text.Trim()

Answer 2

I just stumbled upon this and using @b_stil's snippet, I was able to figure the culprit column. And on futher investigation, I figured i needed to trim the column just like @Liji Chandran suggested but I was using IExcelDataReader and I couldn't figure out an easy way to validate and trim each of my 160 columns.

Then I stumbled upon this class, (ValidatingDataReader) class from CSVReader.

Interesting thing about this class is that it gives you the source and destination columns data length, the culprit row and even the column value that's causing the error.

All I did was just trim all (nvarchar, varchar, char and nchar) columns.

I just changed my GetValue method to this:

 object IDataRecord.GetValue(int i)
    {
        object columnValue = reader.GetValue(i);

        if (i > -1 && i < lookup.Length)
        {
            DataRow columnDef = lookup[i];
            if
            (
                (
                    (string)columnDef["DataTypeName"] == "varchar" ||
                    (string)columnDef["DataTypeName"] == "nvarchar" ||
                    (string)columnDef["DataTypeName"] == "char" ||
                    (string)columnDef["DataTypeName"] == "nchar"
                ) &&
                (
                    columnValue != null &&
                    columnValue != DBNull.Value
                )
            )
            {
                string stringValue = columnValue.ToString().Trim();

                columnValue = stringValue;


                if (stringValue.Length > (int)columnDef["ColumnSize"])
                {
                    string message =
                        "Column value \"" + stringValue.Replace("\"", "\\\"") + "\"" +
                        " with length " + stringValue.Length.ToString("###,##0") +
                        " from source column " + (this as IDataRecord).GetName(i) +
                        " in record " + currentRecord.ToString("###,##0") +
                        " does not fit in destination column " + columnDef["ColumnName"] +
                        " with length " + ((int)columnDef["ColumnSize"]).ToString("###,##0") +
                        " in table " + tableName +
                        " in database " + databaseName +
                        " on server " + serverName + ".";

                    if (ColumnException == null)
                    {
                        throw new Exception(message);
                    }
                    else
                    {
                        ColumnExceptionEventArgs args = new ColumnExceptionEventArgs();

                        args.DataTypeName = (string)columnDef["DataTypeName"];
                        args.DataType = Type.GetType((string)columnDef["DataType"]);
                        args.Value = columnValue;
                        args.SourceIndex = i;
                        args.SourceColumn = reader.GetName(i);
                        args.DestIndex = (int)columnDef["ColumnOrdinal"];
                        args.DestColumn = (string)columnDef["ColumnName"];
                        args.ColumnSize = (int)columnDef["ColumnSize"];
                        args.RecordIndex = currentRecord;
                        args.TableName = tableName;
                        args.DatabaseName = databaseName;
                        args.ServerName = serverName;
                        args.Message = message;

                        ColumnException(args);

                        columnValue = args.Value;
                    }
                }



            }
        }

        return columnValue;
    }

Hope this helps someone

Answer 3

Check the size of the columns in the table you are doing bulk insert/copy. the varchar or other string columns might needs to be extended or the value your are inserting needs to be trim. Column order also should be same as in table.

e.g, Increase size of varchar column 30 to 50 =>

ALTER TABLE [dbo].[TableName] ALTER COLUMN [ColumnName] Varchar(50)

Answer 4

I know this post is old but I ran into this same issue and finally figured out a solution to determine which column was causing the problem and report it back as needed. I determined that colid returned in the SqlException is not zero based so you need to subtract 1 from it to get the value. After that it is used as the index of the _sortedColumnMappings ArrayList of the SqlBulkCopy instance not the index of the column mappings that were added to the SqlBulkCopy instance. One thing to note is that SqlBulkCopy will stop on the first error received so this may not be the only issue but at least helps to figure it out.

try
{
    bulkCopy.WriteToServer(importTable);
    sqlTran.Commit();
}    
catch (SqlException ex)
{
    if (ex.Message.Contains("Received an invalid column length from the bcp client for colid"))
    {
        string pattern = @"\d+";
        Match match = Regex.Match(ex.Message.ToString(), pattern);
        var index = Convert.ToInt32(match.Value) -1;

        FieldInfo fi = typeof(SqlBulkCopy).GetField("_sortedColumnMappings", BindingFlags.NonPublic | BindingFlags.Instance);
        var sortedColumns = fi.GetValue(bulkCopy);
        var items = (Object[])sortedColumns.GetType().GetField("_items", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(sortedColumns);

        FieldInfo itemdata = items[index].GetType().GetField("_metadata", BindingFlags.NonPublic | BindingFlags.Instance);
        var metadata = itemdata.GetValue(items[index]);

        var column = metadata.GetType().GetField("column", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance).GetValue(metadata);
        var length = metadata.GetType().GetField("length", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance).GetValue(metadata);
        throw new DataFormatException(String.Format("Column: {0} contains data with a length greater than: {1}", column, length));
    }

    throw;
}

Answer 5

One of the data columns in the excel (Column Id 6) has one or more cell data that exceed the datacolumn datatype length in the database.

Verify the data in excel. Also verify the data in the excel for its format to be in compliance with the database table schema.

To avoid this, try exceeding the data-length of the string datatype in the database table.

Hope this helps.

Answer 6

Great piece of code, thanks for sharing!

I ended up using reflection to get the actual DataMemberName to throw back to a client on an error (I'm using bulk save in a WCF service). Hopefully someone else will find how I did it useful.

static string GetDataMemberName(string colName, object t) {
  foreach(PropertyInfo propertyInfo in t.GetType().GetProperties()) {
    if (propertyInfo.CanRead) {
      if (propertyInfo.Name == colName) {
        var attributes = propertyInfo.GetCustomAttributes(typeof(DataMemberAttribute), false).FirstOrDefault() as DataMemberAttribute;
        if (attributes != null && !string.IsNullOrEmpty(attributes.Name))
          return attributes.Name;
        return colName;
      }
    }
  }
  return colName;
}