1. 技术文章
  2. Postgres not using index when index scan is much better option

Postgres not using index when index scan is much better option

由 微笑、不失礼 提交于 2019-11-28 08:30:21
Erwin Brandstetter

Index scan -> bitmap index scan -> sequential scan

For few rows it pays to run an index scan. With more rows to return (higher percentage of the table and depending on data distribution, value frequencies and row width) it becomes more likely to find several rows on one data page. Then it pays to switch to a bitmap index scans. Once a large percentage of data pages has to be visited anyway, it's cheaper to run a sequential scan, filter surplus rows and skip the overhead for indexes altogether.

Postgres switches to a sequential scan, expecting to find rows=263962, that's already 3 % of the whole table. (While only rows=47935 are actually found, see below.)

More in this related answer:

Beware of forcing query plans

You cannot force a certain planner method directly in Postgres, but you can make other methods seem extremely expensive for debugging purposes. See Planner Method Configuration in the manual.

SET enable_seqscan = off (like suggested in another answer) does that to sequential scans. But that's intended for debugging purposes in your session only. Do not use this as a general setting in production unless you know exactly what you are doing. It can force ridiculous query plans. Quoting the manual:

These configuration parameters provide a crude method of influencing the query plans chosen by the query optimizer. If the default plan chosen by the optimizer for a particular query is not optimal, a temporary solution is to use one of these configuration parameters to force the optimizer to choose a different plan. Better ways to improve the quality of the plans chosen by the optimizer include adjusting the planer cost constants (see Section 18.7.2), running ANALYZE manually, increasing the value of the default_statistics_target configuration parameter, and increasing the amount of statistics collected for specific columns using ALTER TABLE SET STATISTICS.

That's already most of the advice you need.

In this particular case, Postgres expects 5-6 times more hits on email_activities.email_recipient_id than are actually found:

estimated rows=227007 vs. actual ... rows=40789
estimated rows=263962 vs. actual ... rows=47935

If you run this query often it will pay to have ANALYZE look at a bigger sample for more accurate statistics on the particular column. Your table is big (~ 10M rows), so make that:

ALTER TABLE email_activities ALTER COLUMN email_recipient_id
SET STATISTICS 3000;  -- max 10000, default 100

Then ANALYZE email_activities;

Measure of last resort

In very rare cases you might resort to force an index with SET LOCAL enable_seqscan = off in a separate transaction or in a function with its own environment. Like:

CREATE OR REPLACE FUNCTION f_count_dist_recipients(_email_campaign_id int, _limit int)
  RETURNS bigint AS
$func$
   SELECT COUNT(DISTINCT a.email_recipient_id)
   FROM   email_activities a
   WHERE  a.email_recipient_id IN (
      SELECT id
      FROM   email_recipients
      WHERE  email_campaign_id = $1
      LIMIT  $2)       -- or consider query below
$func$  LANGUAGE sql VOLATILE COST 100000 SET enable_seqscan = off;

The setting only applies to the local scope of the function.

Warning: This is just a proof of concept. Even this much less radical manual intervention might bite you in the long run. Cardinalities, value frequencies, your schema, global Postgres settings, everything changes over time. You are going to upgrade to a new Postgres version. The query plan you force now, may become a very bad idea later.

And typically this is just a workaround for a problem with your setup. Better find and fix it.

Alternative query

Essential information is missing in the question, but this equivalent query is probably faster and more likely to use an index on (email_recipient_id) - increasingly so for a bigger LIMIT.

SELECT COUNT(*) AS ct
FROM  (
   SELECT id
   FROM   email_recipients
   WHERE  email_campaign_id = 1607
   LIMIT  43000
   ) r
WHERE  EXISTS (
   SELECT 1
   FROM   email_activities
   WHERE  email_recipient_id = r.id);

A sequential scan can be more efficient, even when an index exists. In this case, postgres seems to estimate things rather wrong. An ANALYZE <TABLE> on all related tables can help in such cases. If it doesnt, you can set the variable enable_seqscan to OFF, to force postgres to use an index whenever technically possible, at the expense, that sometimes an index-scan will be used when a sequential scan would perform better.

标签
sql
postgresql
postgresql-performance
database-indexes
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!