readme

来自「PostgreSQL7.4.6 for Linux」· 代码 · 共 408 行 · 第 1/2 页

$Header: /cvsroot/pgsql/src/backend/utils/mmgr/README,v 1.4 2003/04/30 19:04:12 tgl Exp $

Notes about memory allocation redesign
--------------------------------------

Up through version 7.0, Postgres had serious problems with memory leakage
during large queries that process a lot of pass-by-reference data.  There
was no provision for recycling memory until end of query.  This needs to be
fixed, even more so with the advent of TOAST which will allow very large
chunks of data to be passed around in the system.  This document describes
the new memory management plan implemented in 7.1.


Background
----------

We already do most of our memory allocation in "memory contexts", which
are usually AllocSets as implemented by backend/utils/mmgr/aset.c.  What
we need to do is create more contexts and define proper rules about when
they can be freed.

The basic operations on a memory context are:

* create a context

* allocate a chunk of memory within a context (equivalent of standard
  C library's malloc())

* delete a context (including freeing all the memory allocated therein)

* reset a context (free all memory allocated in the context, but not the
  context object itself)

Given a chunk of memory previously allocated from a context, one can
free it or reallocate it larger or smaller (corresponding to standard
library's free() and realloc() routines).  These operations return memory
to or get more memory from the same context the chunk was originally
allocated in.

At all times there is a "current" context denoted by the
CurrentMemoryContext global variable.  The backend macro palloc()
implicitly allocates space in that context.  The MemoryContextSwitchTo()
operation selects a new current context (and returns the previous context,
so that the caller can restore the previous context before exiting).

The main advantage of memory contexts over plain use of malloc/free is
that the entire contents of a memory context can be freed easily, without
having to request freeing of each individual chunk within it.  This is
both faster and more reliable than per-chunk bookkeeping.  We already use
this fact to clean up at transaction end: by resetting all the active
contexts, we reclaim all memory.  What we need are additional contexts
that can be reset or deleted at strategic times within a query, such as
after each tuple.


pfree/repalloc no longer depend on CurrentMemoryContext
-------------------------------------------------------

In this proposal, pfree() and repalloc() can be applied to any chunk
whether it belongs to CurrentMemoryContext or not --- the chunk's owning
context will be invoked to handle the operation, regardless.  This is a
change from the old requirement that CurrentMemoryContext must be set
to the same context the memory was allocated from before one can use
pfree() or repalloc().  The old coding requirement is obviously fairly
error-prone, and will become more so the more context-switching we do;
so I think it's essential to use CurrentMemoryContext only for palloc.
We can avoid needing it for pfree/repalloc by putting restrictions on
context managers as discussed below.

We could even consider getting rid of CurrentMemoryContext entirely,
instead requiring the target memory context for allocation to be specified
explicitly.  But I think that would be too much notational overhead ---
we'd have to pass an apppropriate memory context to called routines in
many places.  For example, the copyObject routines would need to be passed
a context, as would function execution routines that return a
pass-by-reference datatype.  And what of routines that temporarily
allocate space internally, but don't return it to their caller?  We
certainly don't want to clutter every call in the system with "here is
a context to use for any temporary memory allocation you might want to
do".  So there'd still need to be a global variable specifying a suitable
temporary-allocation context.  That might as well be CurrentMemoryContext.


Additions to the memory-context mechanism
-----------------------------------------

If we are going to have more contexts, we need more mechanism for keeping
track of them; else we risk leaking whole contexts under error conditions.

We can do this by creating trees of "parent" and "child" contexts.  When
creating a memory context, the new context can be specified to be a child
of some existing context.  A context can have many children, but only one
parent.  In this way the contexts form a forest (not necessarily a single
tree, since there could be more than one top-level context).

We then say that resetting or deleting any particular context resets or
deletes all its direct and indirect children as well.  This feature allows
us to manage a lot of contexts without fear that some will be leaked; we
only need to keep track of one top-level context that we are going to
delete at transaction end, and make sure that any shorter-lived contexts
we create are descendants of that context.  Since the tree can have
multiple levels, we can deal easily with nested lifetimes of storage,
such as per-transaction, per-statement, per-scan, per-tuple.  Storage
lifetimes that only partially overlap can be handled by allocating
from different trees of the context forest (there are some examples
in the next section).

For convenience we will also want operations like "reset/delete all
children of a given context, but don't reset or delete that context
itself".


Globally known contexts
-----------------------

There will be several widely-known contexts that will typically be
referenced through global variables.  At any instant the system may
contain many additional contexts, but all other contexts should be direct
or indirect children of one of these contexts to ensure they are not
leaked in event of an error.

TopMemoryContext --- this is the actual top level of the context tree;
every other context is a direct or indirect child of this one.  Allocating
here is essentially the same as "malloc", because this context will never
be reset or deleted.  This is for stuff that should live forever, or for
stuff that the controlling module will take care of deleting at the
appropriate time.  An example is fd.c's tables of open files, as well as
the context management nodes for memory contexts themselves.  Avoid
allocating stuff here unless really necessary, and especially avoid
running with CurrentMemoryContext pointing here.

PostmasterContext --- this is the postmaster's normal working context.
After a backend is spawned, it can delete PostmasterContext to free its
copy of memory the postmaster was using that it doesn't need.  (Anything
that has to be passed from postmaster to backends will be passed in
TopMemoryContext.  The postmaster will have only TopMemoryContext,
PostmasterContext, and ErrorContext --- the remaining top-level contexts
will be set up in each backend during startup.)

CacheMemoryContext --- permanent storage for relcache, catcache, and
related modules.  This will never be reset or deleted, either, so it's
not truly necessary to distinguish it from TopMemoryContext.  But it
seems worthwhile to maintain the distinction for debugging purposes.
(Note: CacheMemoryContext will have child-contexts with shorter lifespans.
For example, a child context is the best place to keep the subsidiary
storage associated with a relcache entry; that way we can free rule
parsetrees and so forth easily, without having to depend on constructing
a reliable version of freeObject().)

MessageContext --- this context holds the current command message from the
frontend, as well as any derived storage that need only live as long as
the current message (for example, in simple-Query mode the parse and plan
trees can live here).  This context will be reset, and any children
deleted, at the top of each cycle of the outer loop of PostgresMain.  This
is kept separate from per-transaction and per-portal contexts because a
query string might need to live either a longer or shorter time than any
single transaction or portal.

TopTransactionContext --- this holds everything that lives until end of
transaction (longer than one statement within a transaction!).  An example
of what has to be here is the list of pending NOTIFY messages to be sent
at xact commit.  This context will be reset, and all its children deleted,
at conclusion of each transaction cycle.  Note: this context is NOT
cleared immediately upon error; its contents will survive until the
transaction block is exited by COMMIT/ROLLBACK.
(If we ever implement nested transactions, TopTransactionContext may need
to be split into a true "top" pointer and a "current transaction" pointer.)

QueryContext --- this is not actually a separate context, but a global
variable pointing to the context that holds the current command's parse
and plan trees.  (In simple-Query mode this points to MessageContext;
when executing a prepared statement it will point at the prepared
statement's private context.)  Generally it is not appropriate for any
code to use QueryContext as an allocation target --- from the point of
view of any code that would be referencing the QueryContext variable,
it's a read-only context.

PortalContext --- this is not actually a separate context either, but a
global variable pointing to the per-portal context of the currently active
execution portal.  This can be used if it's necessary to allocate storage
that will live just as long as the execution of the current portal requires.

ErrorContext --- this permanent context will be switched into for error
recovery processing, and then reset on completion of recovery.  We'll
arrange to have, say, 8K of memory available in it at all times.  In this
way, we can ensure that some memory is available for error recovery even
if the backend has run out of memory otherwise.  This allows out-of-memory
to be treated as a normal ERROR condition, not a FATAL error.


Contexts for prepared statements and portals
--------------------------------------------

A prepared-statement object has an associated private context, in which
the parse and plan trees for its query are stored.  Because these trees
are read-only to the executor, the prepared statement can be re-used many
times without further copying of these trees.  QueryContext points at this
private context while executing any portal built from the prepared
statement.

An execution-portal object has a private context that is referenced by
PortalContext when the portal is active.  In the case of a portal created
by DECLARE CURSOR, this private context contains the query parse and plan
trees (there being no other object that can hold them).  Portals created