Bulk upload strategy for SQL server

Question

I am uploading data from CSV to SQL table using the following function.

Is there a better way to do it?

I am concerned about , right now, connection hold for lon

Answer 1

You can consider using Cinchoo ETL, an open source library to do the bulk upload CSV file to database.

Option 1:

Load the CSV file straight to database

string connectionstring = @"#YOUR DB ConnectionString#";
using (SqlBulkCopy bcp = new SqlBulkCopy(connectionstring))
{
    using (var p = new ChoCSVReader("#YOUR CSV FILE#"))
    {
        bcp.DestinationTableName = "#TABLENAME#";
        bcp.EnableStreaming = true;
        bcp.BatchSize = 10000;
        bcp.BulkCopyTimeout = 0;
        bcp.NotifyAfter = 100;
        bcp.SqlRowsCopied += delegate (object sender, SqlRowsCopiedEventArgs e)
        {
            Console.WriteLine(e.RowsCopied.ToString("#,##0") + " rows copied.");
        };
        bcp.WriteToServer(p.AsDataReader());
    }
}

Option 2:

If the load of CSV done already and outputted as List, you still can upload them to database as below

List objs = # Your input objects #;

string connectionstring = @"#YOUR DB ConnectionString#";
using (SqlBulkCopy bcp = new SqlBulkCopy(connectionstring))
{
    bcp.DestinationTableName = "#TABLENAME#";
    bcp.EnableStreaming = true;
    bcp.BatchSize = 10000;
    bcp.BulkCopyTimeout = 0;
    bcp.NotifyAfter = 100;
    bcp.SqlRowsCopied += delegate (object sender, SqlRowsCopiedEventArgs e)
    {
        Console.WriteLine(e.RowsCopied.ToString("#,##0") + " rows copied.");
    };
    bcp.WriteToServer(objs.AsDataReader());
}