SQL: Select Keys that doesn't exist in one table

Question

I got a table with a normal setup of auto inc. ids. Some of the rows have been deleted so the ID list could look something like this:

(1, 2, 3, 5, 8,

Answer 1

The problem is that T1 could have a million rows or ten million rows, and that number could change, so you don't know how many rows your comparison table, T2, the one that has no gaps, should have, for doing a WHERE NOT EXISTS or a LEFT JOIN testing for NULL.

But the question is, why do you care if there are missing values? I submit that, when an application is properly architected, it should not matter if there are gaps in an autoincrementing key sequence. Even an application where gaps do matter, such as a check-register, should not be using an autoincrenting primary key as a synonym for the check number.

Care to elaborate on your application requirement?

Answer 2

The function you are looking for is NOT IN (an alias for <> ALL)

The MYSQL documentation:

http://dev.mysql.com/doc/refman/5.0/en/all-subqueries.html

An Example of its use:

http://www.roseindia.net/sql/mysql-example/not-in.shtml

Enjoy!

Answer 3

We can use MYSQL not in option.

SELECT id
FROM table_one
WHERE id NOT IN ( SELECT id FROM table_two )

---

Edited

If you are getting the source from a csv file then you can simply have to put these values directly like:

I am assuming that the CSV are like 1,2,3,...,n

SELECT id
FROM table_one
WHERE id NOT IN ( 1,2,3,...,n );

---

EDIT 2

Or If you want to select the other way around then you can use mysqlimport to import data in temporary table in MySQL Database and retrieve the result and delete the table.

Like:

Create table

CREATE TABLE my_temp_table(
   ids INT,
);

load .csv file

LOAD DATA LOCAL INFILE 'yourIDs.csv' INTO TABLE my_temp_table
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(ids);

Selecting records

SELECT ids FROM my_temp_table
WHERE ids NOT IN ( SELECT id FROM table_one )

dropping table

DROP TABLE IF EXISTS my_temp_table

Answer 4

What about using a left join ; something like this :

select second_table.id
from second_table
    left join first_table on first_table.id = second_table.id
where first_table.is is null

You could also go with a sub-query ; depending on the situation, it might, or might not, be faster, though :

select second_table.id
from second_table
where second_table.id not in (
    select first_table.id
    from first_table
)

Or with a not exists :

select second_table.id
from second_table
where not exists (
    select 1
    from first_table
    where first_table.id = second_table.id
)

Answer 5

OK, I've read your edits/elaboration. Syncrhonizing two databases where the second is not supposed to insert any new rows, but might do so, sounds like a problem waiting to happen.

Neither approach suggested above (WHERE NOT EXISTS or LEFT JOIN) is air-tight and neither is a way to guarantee logical integrity between the two systems. They will not let you know which system created a row in situations where both tables contain a row with the same id. You're focusing on gaps now, but another problem is duplicate ids.

For example, if both tables have a row with id 13887, you cannot assume that database1 created the row. It could have been inserted into database2, and then database1 could insert a new row using that same id. You would have to compare all column values to ascertain that the rows are the same or not.

I'd suggest therefore that you also explore GUID as a replacement for autoincrementing integers. You cannot prevent database2 from inserting rows, but at least with GUIDs you won't run into a problem where the second database has inserted a row and assigned it a primary key value that your first database might also use, resulting in two different rows with the same id. CreationDateTime and LastUpdateDateTime columns would also be useful.

However, a proper solution, if it is available to you, is to maintain just one database and give users remote access to it, for example, via a web interface. That would eliminate the mess and complication of replication/synchronization issues.

If a remote-access web-interface is not feasible, perhaps you could make one of the databases read-only? Or does database2 have to make updates to the rows? Perhaps you could deny insert privilege? What database engine are you using?

Answer 6

I have the same problem: I have a list of values from the user, and I want to find the subset that does not exist in anther table. I did it in oracle by building a pseudo-table in the select statement Here's a way to do it in Oracle. Try it in MySQL without the "from dual":

-- find ids from user (1,2,3) that *don't* exist in my person table
-- build a pseudo table and join it with my person table
select pseudo.id from (
  select '1' as id from dual
  union select '2' as id from dual
  union select '3' as id from dual
) pseudo
left join person
  on person.person_id = pseudo.id
where person.person_id is null