How to avoid inserting duplicate records when using a T-SQL Merge statement

Question

I am attempting to insert many records using T-SQL\'s MERGE statement, but my query fails to INSERT when there are duplicate records in the source table. The failure is caused b

Answer 1

Given the source has duplicates and you aren't using MERGE fully, I'd use an INSERT.

 INSERT dbo.tbl1 (col2,col3) 
 SELECT DISTINCT col2,col3
 FROM #tmp src
 WHERE NOT EXISTS (
       SELECT *
       FROM dbo.tbl1 tbl
       WHERE tbl.col2 = src.col2 AND tbl.col3 = src.col3)

The reason MERGE fails is that it isn't checked row by row. All non-matches are found, then it tries to INSERT all these. It doesn't check for rows in the same batch that already match.

This reminds me a bit of the "Halloween problem" where early data changes of an atomic operation affect later data changes: it isn't correct