indices.sgml

PostgreSQL7.4.6 for Linux

    inherent (due to the nature of the application) and static (not
    changing over time), this is not difficult, but if the common values are
    merely due to the coincidental data load this can require a lot of
    maintenance work.
   </para>
  </example>

  <para>
   Another possibility is to exclude values from the index that the
   typical query workload is not interested in; this is shown in <xref
   linkend="indexes-partial-ex2">.  This results in the same
   advantages as listed above, but it prevents the
   <quote>uninteresting</quote> values from being accessed via that
   index at all, even if an index scan might be profitable in that
   case.  Obviously, setting up partial indexes for this kind of
   scenario will require a lot of care and experimentation.
  </para>

  <example id="indexes-partial-ex2">
   <title>Setting up a Partial Index to Exclude Uninteresting Values</title>

   <para>
    If you have a table that contains both billed and unbilled orders,
    where the unbilled orders take up a small fraction of the total
    table and yet those are the most-accessed rows, you can improve
    performance by creating an index on just the unbilled rows.  The
    command to create the index would look like this:
<programlisting>
CREATE INDEX orders_unbilled_index ON orders (order_nr)
    WHERE billed is not true;
</programlisting>
   </para>

   <para>
    A possible query to use this index would be
<programlisting>
SELECT * FROM orders WHERE billed is not true AND order_nr < 10000;
</programlisting>
    However, the index can also be used in queries that do not involve
    <structfield>order_nr</> at all, e.g.,
<programlisting>
SELECT * FROM orders WHERE billed is not true AND amount > 5000.00;
</programlisting>
    This is not as efficient as a partial index on the
    <structfield>amount</> column would be, since the system has to
    scan the entire index.  Yet, if there are relatively few unbilled
    orders, using this partial index just to find the unbilled orders
    could be a win.
   </para>

   <para>
    Note that this query cannot use this index:
<programlisting>
SELECT * FROM orders WHERE order_nr = 3501;
</programlisting>
    The order 3501 may be among the billed or among the unbilled
    orders.
   </para>
  </example>

  <para>
   <xref linkend="indexes-partial-ex2"> also illustrates that the
   indexed column and the column used in the predicate do not need to
   match.  <productname>PostgreSQL</productname> supports partial
   indexes with arbitrary predicates, so long as only columns of the
   table being indexed are involved.  However, keep in mind that the
   predicate must match the conditions used in the queries that
   are supposed to benefit from the index.  To be precise, a partial
   index can be used in a query only if the system can recognize that
   the <literal>WHERE</> condition of the query mathematically implies
   the predicate of the index.
   <productname>PostgreSQL</productname> does not have a sophisticated
   theorem prover that can recognize mathematically equivalent
   expressions that are written in different forms.  (Not
   only is such a general theorem prover extremely difficult to
   create, it would probably be too slow to be of any real use.)
   The system can recognize simple inequality implications, for example
   <quote>x &lt; 1</quote> implies <quote>x &lt; 2</quote>; otherwise
   the predicate condition must exactly match part of the query's
   <literal>WHERE</> condition
   or the index will not be recognized to be usable.
  </para>

  <para>
   A third possible use for partial indexes does not require the
   index to be used in queries at all.  The idea here is to create
   a unique index over a subset of a table, as in <xref
   linkend="indexes-partial-ex3">.  This enforces uniqueness
   among the rows that satisfy the index predicate, without constraining
   those that do not.
  </para>

  <example id="indexes-partial-ex3">
   <title>Setting up a Partial Unique Index</title>

   <para>
    Suppose that we have a table describing test outcomes.  We wish
    to ensure that there is only one <quote>successful</> entry for
    a given subject and target combination, but there might be any number of
    <quote>unsuccessful</> entries.  Here is one way to do it:
<programlisting>
CREATE TABLE tests (
    subject text,
    target text,
    success boolean,
    ...
);

CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target)
    WHERE success;
</programlisting>
    This is a particularly efficient way of doing it when there are few
    successful tests and many unsuccessful ones.
   </para>
  </example>

  <para>
   Finally, a partial index can also be used to override the system's
   query plan choices.  It may occur that data sets with peculiar
   distributions will cause the system to use an index when it really
   should not.  In that case the index can be set up so that it is not
   available for the offending query.  Normally,
   <productname>PostgreSQL</> makes reasonable choices about index
   usage (e.g., it avoids them when retrieving common values, so the
   earlier example really only saves index size, it is not required to
   avoid index usage), and grossly incorrect plan choices are cause
   for a bug report.
  </para>

  <para>
   Keep in mind that setting up a partial index indicates that you
   know at least as much as the query planner knows, in particular you
   know when an index might be profitable.  Forming this knowledge
   requires experience and understanding of how indexes in
   <productname>PostgreSQL</> work.  In most cases, the advantage of a
   partial index over a regular index will not be much.
  </para>

  <para>
   More information about partial indexes can be found in <xref
   linkend="STON89b">, <xref linkend="OLSON93">, and <xref
   linkend="SESHADRI95">.
  </para>
 </sect1>

 <sect1 id="indexes-examine">
  <title>Examining Index Usage</title>

  <indexterm zone="indexes-examine">
   <primary>index</primary>
   <secondary>examining usage</secondary>
  </indexterm>

  <para>
   Although indexes in <productname>PostgreSQL</> do not need
   maintenance and tuning, it is still important to check
   which indexes are actually used by the real-life query workload.
   Examining index usage for an individual query is done with the
   <xref linkend="sql-explain" endterm="sql-explain-title">
   command; its application for this purpose is
   illustrated in <xref linkend="using-explain">.
   It is also possible to gather overall statistics about index usage
   in a running server, as described in <xref linkend="monitoring-stats">.
  </para>

  <para>
   It is difficult to formulate a general procedure for determining
   which indexes to set up.  There are a number of typical cases that
   have been shown in the examples throughout the previous sections.
   A good deal of experimentation will be necessary in most cases.
   The rest of this section gives some tips for that.
  </para>

  <itemizedlist>
   <listitem>
    <para>
     Always run <xref linkend="sql-analyze" endterm="sql-analyze-title">
     first.  This command
     collects statistics about the distribution of the values in the
     table.  This information is required to guess the number of rows
     returned by a query, which is needed by the planner to assign
     realistic costs to each possible query plan.  In absence of any
     real statistics, some default values are assumed, which are
     almost certain to be inaccurate.  Examining an application's
     index usage without having run <command>ANALYZE</command> is
     therefore a lost cause.
    </para>
   </listitem>

   <listitem>
    <para>
     Use real data for experimentation.  Using test data for setting
     up indexes will tell you what indexes you need for the test data,
     but that is all.
    </para>

    <para>
     It is especially fatal to use proportionally reduced data sets.
     While selecting 1000 out of 100000 rows could be a candidate for
     an index, selecting 1 out of 100 rows will hardly be, because the
     100 rows will probably fit within a single disk page, and there
     is no plan that can beat sequentially fetching 1 disk page.
    </para>

    <para>
     Also be careful when making up test data, which is often
     unavoidable when the application is not in production use yet.
     Values that are very similar, completely random, or inserted in
     sorted order will skew the statistics away from the distribution
     that real data would have.
    </para>
   </listitem>

   <listitem>
    <para>
     When indexes are not used, it can be useful for testing to force
     their use.  There are run-time parameters that can turn off
     various plan types (described in <xref linkend="runtime-config">).
     For instance, turning off sequential scans
     (<varname>enable_seqscan</>) and nested-loop joins
     (<varname>enable_nestloop</>), which are the most basic plans,
     will force the system to use a different plan.  If the system
     still chooses a sequential scan or nested-loop join then there is
     probably a more fundamental problem for why the index is not
     used, for example, the query condition does not match the index.
     (What kind of query can use what kind of index is explained in
     the previous sections.)
    </para>
   </listitem>

   <listitem>
    <para>
     If forcing index usage does use the index, then there are two
     possibilities: Either the system is right and using the index is
     indeed not appropriate, or the cost estimates of the query plans
     are not reflecting reality.  So you should time your query with
     and without indexes.  The <command>EXPLAIN ANALYZE</command>
     command can be useful here.
    </para>
   </listitem>

   <listitem>
    <para>
     If it turns out that the cost estimates are wrong, there are,
     again, two possibilities.  The total cost is computed from the
     per-row costs of each plan node times the selectivity estimate of
     the plan node.  The costs of the plan nodes can be tuned with
     run-time parameters (described in <xref linkend="runtime-config">).
     An inaccurate selectivity estimate is due to
     insufficient statistics.  It may be possible to help this by
     tuning the statistics-gathering parameters (see
     <xref linkend="sql-altertable" endterm="sql-altertable-title">).
    </para>

    <para>
     If you do not succeed in adjusting the costs to be more
     appropriate, then you may have to resort to forcing index usage
     explicitly.  You may also want to contact the
     <productname>PostgreSQL</> developers to examine the issue.
    </para>
   </listitem>
  </itemizedlist>
 </sect1>
</chapter>

<!-- Keep this comment at the end of the file
Local variables:
mode:sgml
sgml-omittag:nil
sgml-shorttag:t
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
sgml-parent-document:nil
sgml-default-dtd-file:"./reference.ced"
sgml-exposed-tags:nil
sgml-local-catalogs:("/usr/lib/sgml/catalog")
sgml-local-ecat-files:nil
End:
-->