readme

来自「PostgreSQL7.4.6 for Linux」· 代码 · 共 408 行 · 第 1/2 页

from prepared statements simply reference the prepared statements' trees,
and won't actually need any storage allocated in their private contexts.


Transient contexts during execution
-----------------------------------

When creating a prepared statement, the parse and plan trees will be built
in a temporary context that's a child of MessageContext (so that it will
go away automatically upon error).  On success, the finished plan is
copied to the prepared statement's private context, and the temp context
is released; this allows planner temporary space to be recovered before
execution begins.  (In simple-Query mode we'll not bother with the extra
copy step, so the planner temp space stays around till end of query.)

The top-level executor routines, as well as most of the "plan node"
execution code, will normally run in a context that is created by
ExecutorStart and destroyed by ExecutorEnd; this context also holds the
"plan state" tree built during ExecutorStart.  Most of the memory
allocated in these routines is intended to live until end of query,
so this is appropriate for those purposes.  The executor's top context
is a child of PortalContext, that is, the per-portal context of the
portal that represents the query's execution.

The main improvement needed in the executor is that expression evaluation
--- both for qual testing and for computation of targetlist entries ---
needs to not leak memory.  To do this, each ExprContext (expression-eval
context) created in the executor will now have a private memory context
associated with it, and we'll arrange to switch into that context when
evaluating expressions in that ExprContext.  The plan node that owns the
ExprContext is responsible for resetting the private context to empty
when it no longer needs the results of expression evaluations.  Typically
the reset is done at the start of each tuple-fetch cycle in the plan node.

Note that this design gives each plan node its own expression-eval memory
context.  This appears necessary to handle nested joins properly, since
an outer plan node might need to retain expression results it has computed
while obtaining the next tuple from an inner node --- but the inner node
might execute many tuple cycles and many expressions before returning a
tuple.  The inner node must be able to reset its own expression context
more often than once per outer tuple cycle.  Fortunately, memory contexts
are cheap enough that giving one to each plan node doesn't seem like a
problem.

A problem with running index accesses and sorts in a query-lifespan context
is that these operations invoke datatype-specific comparison functions,
and if the comparators leak any memory then that memory won't be recovered
till end of query.  The comparator functions all return bool or int32,
so there's no problem with their result data, but there can be a problem
with leakage of internal temporary data.  In particular, comparator
functions that operate on TOAST-able data types will need to be careful
not to leak detoasted versions of their inputs.  This is annoying, but
it appears a lot easier to make the comparators conform than to fix the
index and sort routines, so that's what I propose to do for 7.1.  Further
cleanup can be left for another day.

There will be some special cases, such as aggregate functions.  nodeAgg.c
needs to remember the results of evaluation of aggregate transition
functions from one tuple cycle to the next, so it can't just discard
all per-tuple state in each cycle.  The easiest way to handle this seems
to be to have two per-tuple contexts in an aggregate node, and to
ping-pong between them, so that at each tuple one is the active allocation
context and the other holds any results allocated by the prior cycle's
transition function.

Executor routines that switch the active CurrentMemoryContext may need
to copy data into their caller's current memory context before returning.
I think there will be relatively little need for that, because of the
convention of resetting the per-tuple context at the *start* of an
execution cycle rather than at its end.  With that rule, an execution
node can return a tuple that is palloc'd in its per-tuple context, and
the tuple will remain good until the node is called for another tuple
or told to end execution.  This is pretty much the same state of affairs
that exists now, since a scan node can return a direct pointer to a tuple
in a disk buffer that is only guaranteed to remain good that long.

A more common reason for copying data will be to transfer a result from
per-tuple context to per-run context; for example, a Unique node will
save the last distinct tuple value in its per-run context, requiring a
copy step.

Another interesting special case is VACUUM, which needs to allocate
working space that will survive its forced transaction commits, yet
be released on error.  Currently it does that through a "portal",
which is essentially a child context of TopMemoryContext.  While that
way still works, it's ugly since xact abort needs special processing
to delete the portal.  Better would be to use a context that's a child
of PortalContext and hence is certain to go away as part of normal
processing.  (Eventually we might have an even better solution from
nested transactions, but this'll do fine for now.)


Mechanisms to allow multiple types of contexts
----------------------------------------------

We may want several different types of memory contexts with different
allocation policies but similar external behavior.  To handle this,
memory allocation functions will be accessed via function pointers,
and we will require all context types to obey the conventions given here.
(This is not very far different from the existing code.)

A memory context will be represented by an object like

typedef struct MemoryContextData
{
    NodeTag        type;           /* identifies exact kind of context */
    MemoryContextMethods methods;
    MemoryContextData *parent;     /* NULL if no parent (toplevel context) */
    MemoryContextData *firstchild; /* head of linked list of children */
    MemoryContextData *nextchild;  /* next child of same parent */
    char          *name;           /* context name (just for debugging) */
} MemoryContextData, *MemoryContext;

This is essentially an abstract superclass, and the "methods" pointer is
its virtual function table.  Specific memory context types will use
derived structs having these fields as their first fields.  All the
contexts of a specific type will have methods pointers that point to the
same static table of function pointers, which will look like

typedef struct MemoryContextMethodsData
{
    Pointer     (*alloc) (MemoryContext c, Size size);
    void        (*free_p) (Pointer chunk);
    Pointer     (*realloc) (Pointer chunk, Size newsize);
    void        (*reset) (MemoryContext c);
    void        (*delete) (MemoryContext c);
} MemoryContextMethodsData, *MemoryContextMethods;

Alloc, reset, and delete requests will take a MemoryContext pointer
as parameter, so they'll have no trouble finding the method pointer
to call.  Free and realloc are trickier.  To make those work, we will
require all memory context types to produce allocated chunks that
are immediately preceded by a standard chunk header, which has the
layout

typedef struct StandardChunkHeader
{
    MemoryContext mycontext;         /* Link to owning context object */
    Size          size;              /* Allocated size of chunk */
};

It turns out that the existing aset.c memory context type does this
already, and probably any other kind of context would need to have the
same data available to support realloc, so this is not really creating
any additional overhead.  (Note that if a context type needs more per-
allocated-chunk information than this, it can make an additional
nonstandard header that precedes the standard header.  So we're not
constraining context-type designers very much.)

Given this, the pfree routine will look something like

    StandardChunkHeader * header = 
        (StandardChunkHeader *) ((char *) p - sizeof(StandardChunkHeader));

    (*header->mycontext->methods->free_p) (p);

We could do it as a macro, but the macro would have to evaluate its
argument twice, which seems like a bad idea (the current pfree macro
does not do that).  This is already saving two levels of function call
compared to the existing code, so I think we're doing fine without
squeezing out that last little bit ...


More control over aset.c behavior
---------------------------------

Currently, aset.c allocates an 8K block upon the first allocation in
a context, and doubles that size for each successive block request.
That's good behavior for a context that might hold *lots* of data, and
the overhead wasn't bad when we had only a few contexts in existence.
With dozens if not hundreds of smaller contexts in the system, we will
want to be able to fine-tune things a little better.

The creator of a context will be able to specify an initial block size
and a maximum block size.  Selecting smaller values will prevent wastage
of space in contexts that aren't expected to hold very much (an example is
the relcache's per-relation contexts).

Also, it will be possible to specify a minimum context size.  If this
value is greater than zero then a block of that size will be grabbed
immediately upon context creation, and cleared but not released during
context resets.  This feature is needed for ErrorContext (see above),
but will most likely not be used for other contexts.

We expect that per-tuple contexts will be reset frequently and typically
will not allocate very much space per tuple cycle.  To make this usage
pattern cheap, the first block allocated in a context is not given
back to malloc() during reset, but just cleared.  This avoids malloc
thrashing.


Other notes
-----------

The original version of this proposal suggested that functions returning
pass-by-reference datatypes should be required to return a value freshly
palloc'd in their caller's memory context, never a pointer to an input
value.  I've abandoned that notion since it clearly is prone to error.
In the current proposal, it is possible to discover which context a
chunk of memory is allocated in (by checking the required standard chunk
header), so nodeAgg can determine whether or not it's safe to reset
its working context; it doesn't have to rely on the transition function
to do what it's expecting.