readme.hot

postgresql8.3.4源码,开源数据库

case, and to keep various work arrays as well as the bitmaps in bitmap
scans reasonably sized, the maximum number of line pointers per page
is arbitrarily capped at MaxHeapTuplesPerPage (the most tuples that
could fit without HOT pruning).


VACUUM
------

There is little change to regular vacuum.  It performs pruning to remove
dead heap-only tuples, and cleans up any dead line pointers as if they were
regular dead tuples.


VACUUM FULL
-----------

VACUUM FULL performs an extra operation of collapsing out redirecting line
pointers, by moving the first non-DEAD tuple of each HOT chain to the root
position and clearing its heap-only-tuple flag.  This effectively changes
the user-visible CTID of that tuple.  This would be completely unsafe
during normal concurrent operation, but since VACUUM FULL takes full
exclusive lock on the table, it should be OK.  (Note that VACUUM FULL has
always felt free to change tuples' CTIDs by moving them across pages.)
Eliminating redirection links means that the main body of VACUUM FULL
doesn't have to deal with them, which seems a good thing since VACUUM FULL
is horrendously complex already.

When VACUUM FULL tries to move tuple chains, it does not distinguish regular
and heap-only tuples, but just moves both types the same.  This is OK because
it will move the entire non-DEAD tail of an update chain and remove index
entries for each item moved.  At worst, we'll uselessly search for index
entries matching the heap-only tuples included in the move.


Statistics
----------

Currently, we count HOT updates the same as cold updates for statistics
purposes, though there is an additional per-table counter that counts
only HOT updates.  When a page pruning operation is able to remove a
physical tuple by eliminating an intermediate heap-only tuple or
replacing a physical root tuple by a redirect pointer, a decrement in
the table's number of dead tuples is reported to pgstats, which may
postpone autovacuuming.  Note that we do not count replacing a root tuple
by a DEAD item pointer as decrementing n_dead_tuples; we still want
autovacuum to run to clean up the index entries and DEAD item.

This area probably needs further work ...


CREATE INDEX
------------

CREATE INDEX presents a problem for HOT updates.  While the existing HOT
chains all have the same index values for existing indexes, the columns
in the new index might change within a pre-existing HOT chain, creating
a "broken" chain that can't be indexed properly.

To address this issue, regular (non-concurrent) CREATE INDEX makes the
new index usable only by transactions newer than the CREATE INDEX
command.  This prevents transactions that can see the inconsistent HOT
chains from trying to use the new index and getting incorrect results.  
New transactions can only see the rows visible after the index was
created, hence the HOT chains are consistent for them.

Entries in the new index point to root tuples (tuples with current index
pointers) so that our index uses the same index pointers as all other
indexes on the table.  However the row we want to index is actually at
the *end* of the chain, ie, the most recent live tuple on the HOT chain.
That is the one we compute the index entry values for, but the TID
we put into the index is that of the root tuple.  Since transactions that
will be allowed to use the new index cannot see any of the older tuple
versions in the chain, the fact that they might not match the index entry
isn't a problem.  (Such transactions will check the tuple visibility
information of the older versions and ignore them, without ever looking at
their contents, so the content inconsistency is OK.)  Subsequent updates
to the live tuple will be allowed to extend the HOT chain only if they are
HOT-safe for all the indexes.

Because we have ShareLock on the table, any DELETE_IN_PROGRESS or
INSERT_IN_PROGRESS tuples should have come from our own transaction.
Therefore we can consider them committed since if the CREATE INDEX
commits, they will be committed, and if it aborts the index is discarded.
An exception to this is that early lock release is customary for system
catalog updates, and so we might find such tuples when reindexing a system
catalog.  In that case we deal with it by waiting for the source
transaction to commit or roll back.  (We could do that for user tables
too, but since the case is unexpected we prefer to throw an error.)

Practically, we prevent old transactions from using the new index by
setting pg_index.indcheckxmin to TRUE.  Queries are allowed to use such an
index only after pg_index.xmin is below their TransactionXmin horizon,
thereby ensuring that any incompatible rows in HOT chains are dead to them.
(pg_index.xmin will be the XID of the CREATE INDEX transaction.  The reason
for using xmin rather than a normal column is that the regular vacuum
freezing mechanism will take care of converting xmin to FrozenTransactionId
before it can wrap around.)

This means in particular that the transaction creating the index will be
unable to use the index.  We alleviate that problem somewhat by not setting
indcheckxmin unless the table actually contains HOT chains with
RECENTLY_DEAD members.  (In 8.4 we may be able to improve the situation,
at least for non-serializable transactions, because we expect to be able to
advance TransactionXmin intratransaction.)

Another unpleasant consequence is that it is now risky to use SnapshotAny
in an index scan: if the index was created more recently than the last
vacuum, it's possible that some of the visited tuples do not match the
index entry they are linked to.  This does not seem to be a fatal
objection, since there are few users of SnapshotAny and most use seqscans.
The only exception at this writing is CLUSTER, which is okay because it
does not require perfect ordering of the indexscan readout (and especially
so because CLUSTER tends to write recently-dead tuples out of order anyway).


CREATE INDEX CONCURRENTLY
-------------------------

In the concurrent case we must take a different approach.  We create the
pg_index entry immediately, before we scan the table.  The pg_index entry
is marked as "not ready for inserts".  Then we commit and wait for any
transactions which have the table open to finish.  This ensures that no
new HOT updates will change the key value for our new index, because all
transactions will see the existence of the index and will respect its
constraint on which updates can be HOT.  Other transactions must include
such an index when determining HOT-safety of updates, even though they
must ignore it for both insertion and searching purposes.

We must do this to avoid making incorrect index entries.  For example,
suppose we are building an index on column X and we make an index entry for
a non-HOT tuple with X=1.  Then some other backend, unaware that X is an
indexed column, HOT-updates the row to have X=2, and commits.  We now have
an index entry for X=1 pointing at a HOT chain whose live row has X=2.
We could make an index entry with X=2 during the validation pass, but
there is no nice way to get rid of the wrong entry with X=1.  So we must
have the HOT-safety property enforced before we start to build the new
index.

After waiting for transactions which had the table open, we build the index
for all rows that are valid in a fresh snapshot.  Any tuples visible in the
snapshot will have only valid forward-growing HOT chains.  (They might have
older HOT updates behind them which are broken, but this is OK for the same
reason it's OK in a regular index build.)  As above, we point the index
entry at the root of the HOT-update chain but we use the key value from the
live tuple.

We mark the index open for inserts (but still not ready for reads) then
we again wait for transactions which have the table open.  Then we take
a second reference snapshot and validate the index.  This searches for
tuples missing from the index, and inserts any missing ones.  Again,
the index entries have to have TIDs equal to HOT-chain root TIDs, but
the value to be inserted is the one from the live tuple.

Then we wait until every transaction that could have a snapshot older than
the second reference snapshot is finished.  This ensures that nobody is
alive any longer who could need to see any tuples that might be missing
from the index, as well as ensuring that no one can see any inconsistent
rows in a broken HOT chain (the first condition is stronger than the
second).  Finally, we can mark the index valid for searches.


Limitations and Restrictions
----------------------------

It is worth noting that HOT forever forecloses alternative approaches
to vacuuming, specifically the recompute-the-index-keys approach alluded
to in Technical Challenges above.  It'll be tough to recompute the index
keys for a root line pointer you don't have data for anymore ...


Glossary
--------

Broken HOT Chain

	A HOT chain in which the key value for an index has changed.

	This is not allowed to occur normally but if a new index is created
	it can happen.  In that case various strategies are used to ensure
	that no transaction for which the older tuples are visible can
	use the index.

Cold update

	A normal, non-HOT update, in which index entries are made for
	the new version of the tuple.

Dead line pointer

	A stub line pointer, that does not point to anything, but cannot
	be removed or reused yet because there are index pointers to it.
	Semantically same as a dead tuple.  It has state LP_DEAD.

Heap-only tuple

	A heap tuple with no index pointers, which can only be reached
	from indexes indirectly through its ancestral root tuple.
	Marked with HEAP_ONLY_TUPLE flag.

HOT-safe

	A proposed tuple update is said to be HOT-safe if it changes
	none of the tuple's indexed columns.  It will only become an
	actual HOT update if we can find room on the same page for
	the new tuple version.

HOT update

	An UPDATE where the new tuple becomes a heap-only tuple, and no
	new index entries are made.

HOT-updated tuple

	An updated tuple, for which the next tuple in the chain is a
	heap-only tuple.  Marked with HEAP_HOT_UPDATED flag.

Indexed column

	A column used in an index definition.  The column might not
	actually be stored in the index --- it could be used in a
	functional index's expression, or used in a partial index
	predicate.  HOT treats all these cases alike.

Redirecting line pointer

	A line pointer that points to another line pointer and has no
	associated tuple.  It has the special lp_flags state LP_REDIRECT,
	and lp_off is the OffsetNumber of the line pointer it links to.
	This is used when a root tuple becomes dead but we cannot prune
	the line pointer because there are non-dead heap-only tuples
	further down the chain.

Root tuple

	The first tuple in a HOT update chain; the one that indexes point to.

Update chain

	A chain of updated tuples, in which each tuple's ctid points to
	the next tuple in the chain. A HOT update chain is an update chain
	(or portion of an update chain) that consists of a root tuple and
	one or more heap-only tuples.  A complete update chain can contain
	both HOT and non-HOT (cold) updated tuples.