Any way to SQLBulkCopy “insert or update if exists”?

Question

I need to update a very large table periodically and SQLBulkCopy is perfect for that, only that I have a 2-columns index that prevents duplicates. Is there a way to use SQLB

Answer 1

Instead of create a new temporary table, which BTW consume more space and memory.

I created a Trigger with INSTEAD OF INSERT and use inside MERGE statement.

But don't forget add the parameter SqlBulkCopyOptions.FireTriggers in the SqlBulkCopy.

This is my two cents.

Answer 2

Got a hint from @Ivan. For those who might need, here's what I did.

create trigger yourschma.Tr_your_triger_name
    on yourschma.yourtable
    instead of INSERT
    as
    merge into yourschma.yourtable as target
    using inserted as source
    on (target.yourtableID = source.yourtableID)
    when matched then
        update
        set target.ID     = source.ID,
            target.some_column = source.some_column,
            target.Amount                       = source.Amount
    when not matched by target then
        insert (some_column, Amount)
        values (source.some_column, source.Amount);
go

Answer 3

I would bulk load data into a temporary staging table, then do an upsert into the final table. See http://www.databasejournal.com/features/mssql/article.php/3739131/UPSERT-Functionality-in-SQL-Server-2008.htm for an example of doing an upsert.

Answer 4

Not in one step, but in SQL Server 2008, you could:

• bulk load into staging table
• apply a MERGE statement to update/insert into your real table

Read more about the MERGE statement

Answer 5

Another alternative would be to not use a temporary table but use a stored procedure with a table valued parameter. Pass a datatable to the sp and do the merge there.

Answer 6

I published a nuget package (SqlBulkTools) to solve this problem.

Here's a code example that would achieve a bulk upsert.

var bulk = new BulkOperations();
var books = GetBooks();

using (TransactionScope trans = new TransactionScope())
{
    using (SqlConnection conn = new SqlConnection(ConfigurationManager
    .ConnectionStrings["SqlBulkToolsTest"].ConnectionString))
    {
        bulk.Setup<Book>()
            .ForCollection(books)
            .WithTable("Books")
            .AddAllColumns()
            .BulkInsertOrUpdate()
            .MatchTargetOn(x => x.ISBN)
            .Commit(conn);
    }

    trans.Complete();
}

For very large tables, there are options to add table locks and temporarily disable non-clustered indexes. See SqlBulkTools Documentation for more examples.