sql insert into table from select without duplicates (need more then a DISTINCT)

Question

I am selecting multiple rows and inserting them into another table. I want to make sure that it doesn\'t already exists in the table I am inserting multiple rows into.

Answer 1

pseudo code for what might work

    insert into <target_table> select col1 etc 
from <source_table> 
where <target_table>.keycol not in 
(select source_table.keycol from source_table)

Answer 2

If you already have a unique index on whatever fields need to be unique in the destination table, you can just use INSERT IGNORE (here's the official documentation - the relevant bit is toward the end), and have MySQL throw away the duplicates for you.

Hope this helps!

Answer 3

There are a few MSDN articles out there about this, but by far this one is the best:

http://msdn.microsoft.com/en-us/library/ms162773.aspx

They made it real easy to implement and my problem is now fixed. Also the GUI is ugly, but you actually can set minute intervals without using the command line in windows 2003.

Answer 4

So you're looking to retrieve all unique rows from source table which do not already exist in target table?

SELECT DISTINCT(*) FROM source
WHERE primaryKey NOT IN (SELECT primaryKey FROM target)

That's assuming you have a primary key which you can base the uniqueness on... otherwise, you'll have to check each column for uniqueness.

Answer 5

INSERT INTO target_table (col1, col2, col3)
SELECT DISTINCT st.col1, st.col2, st.col3
FROM source_table st
WHERE NOT EXISTS (SELECT 1 
                  FROM target_table t2
                  WHERE t2.col1 = st.col1 
                    AND t2.col2 = st.col2
                    AND t2.col3 = st.col3)

If the distinct should only be on certain columns (e.g. col1, col2) but you need to insert all column, you will probably need some derived table (ANSI SQL):

INSERT INTO target_table (col1, col2, col3)
SELECT st.col1, st.col2, st.col3
FROM ( 
     SELECT col1, 
            col2, 
            col3, 
            row_number() over (partition by col1, col2 order by col1, col2) as rn
     FROM source_table 
) st
WHERE st.rn = 1
AND NOT EXISTS (SELECT 1 
                FROM target_table t2
                WHERE t2.col1 = st.col1 
                  AND t2.col2 = st.col2)