How to do a Bulk Insert — Linq to Entities

Question

I cannot find any examples on how to do a Bulk/batch insert using Linq to Entities. Do you guys know how to do a Bulk Insert?

Answer 1

I wrote a class that will bulk insert EF entities (or any kind of object as long as the property names match the column names).

The class supports customizing the batch size, pre & post insert events, queued inserts, and "firehose mode" (give it a billion objects, it will respect the batch size).

• The code: BulkInserter<T>
• How to use it

Answer 2

For inserting a huge amount of data in a database, I used to collect all the inserting information into a list and convert this list into a DataTable. I then insert that list to a database via SqlBulkCopy.

Where I send my generated list
LiMyList
which contain information of all bulk data which I want to insert to database
and pass it to my bulk insertion operation

InsertData(LiMyList, "MyTable");

Where InsertData is

 public static void InsertData<T>(List<T> list,string TabelName)
        {
                DataTable dt = new DataTable("MyTable");
                clsBulkOperation blk = new clsBulkOperation();
                dt = ConvertToDataTable(list);
                ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoamingAndLocal);
                using (SqlBulkCopy bulkcopy = new SqlBulkCopy(ConfigurationManager.ConnectionStrings["SchoolSoulDataEntitiesForReport"].ConnectionString))
                {
                    bulkcopy.BulkCopyTimeout = 660;
                    bulkcopy.DestinationTableName = TabelName;
                    bulkcopy.WriteToServer(dt);
                }
        }    

public static DataTable ConvertToDataTable<T>(IList<T> data)
        {
            PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T));
            DataTable table = new DataTable();
            foreach (PropertyDescriptor prop in properties)
                table.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType);
            foreach (T item in data)
            {
                DataRow row = table.NewRow();
                foreach (PropertyDescriptor prop in properties)
                    row[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
                table.Rows.Add(row);
            }
            return table;
        }

Answer 3

For a perfect example of how to do bulk inserts with LINQ to Entities, see http://archive.msdn.microsoft.com/LinqEntityDataReader. It is a wrapper that allows easy use of SqlBulkCopy.

@Marc Gravell is correct, sometimes you have to mix models to get the job done.

Answer 4

Sometimes you simply have to mix models. Perhaps use SqlBulkCopy for this part of your repository (since this plugs directly into the bulk-copy API), and Entity Framework for some of the rest. And if necessary, a bit of direct ADO.NET. Ultimately the goal is to get the job done.