Why using IN(…) when selecting on indexed fields, will kill the performance of SELECT query?

Question

Avoid using IN(...) when selecting on indexed fields, It will kill the performance of SELECT query.

I found this here: https://wikis.ora

Answer 1

Because MySQL can't optimize it.

Here is an example:

explain select * from keywordmaster where id in (1, 567899);

plan (sorry for external link. Doesn't show correctly here)

here is another query:

   explain 
   select * from table where id = 1
   union
   select * from keywordmaster where id = 567899

plan

As you can see in the second query we get ref as const and type is const instead of range. MySQL can't optimize range scans.

Answer 2

I believe IN is treated the same as a group of ORs, so using ORs won't help.

An alternative is to create a temporary table to hold the values of your IN-clause and then join with that temporary table in your SELECT.

For example:

CREATE TEMPORARY TABLE temp_table (v VARCHAR)

INSERT INTO temp_table  VALUES ('foo')
INSERT INTO temp_table  VALUES ('bar')

SELECT * FROM temp_table tmp, orig_table orig
WHERE temp_table.v = orig.value

DROP TEMPORARY TABLE temp_table

Answer 3

To tell the truth, that statement contradicts to many hints that I have read in books and articles on MySQL.

Here is an example: http://www.mysqlperformanceblog.com/2010/01/09/getting-around-optimizer-limitations-with-an-in-list/

Moreover, expr IN(value, ...) itself has additional enhancements for dealing with large value lists, since it is supposed to be used as a useful alternative to certain range queries:

If all values are constants, they are evaluated according to the type of expr and sorted. The search for the item then is done using a binary search. This means IN is very quick if the IN value list consists entirely of constants.

Still overusing INs may result in slow queries. Some cases are noted in the article.

Answer 4

Prior to MySQL 5.0 it seems that mySQL would only use a single index for a table. So, if you had a SELECT * FROM tbl WHERE (a = 6 OR b = 33) it could chooose to use either the a index or the b index, but not both. Note that it says fields, plural. I suspect the advice comes from that time and the work-around was to union the OR results, like so:

SELECT * FROM tbl WHERE (a = 6)
UNION
SELECT * FROM tbl WHERE (b = 33)