malloc.html

sqlite的帮助文档

the test suite provides over 99% statement test coverage.  This is
strong evidence that dynamic memory allocation is used correctly
everywhere within SQLite.</p>

<a name="config"></a>
<h2>3.0 Configuration</h2>

<p>The default memory allocation settings in SQLite are appropriate
for most applications.  However, applications with unusual or particularly
strict requirements may want to adjust the configuration to more closely
align SQLite to their needs.
Both compile-time and start-time configuration options are available.</p>

<a name="altalloc"></a>
<h3>3.1 Alternative low-level memory allocators</h3>

<p>The SQLite source code includes several different memory allocation
modules that can be selected at compile-time, or to a limited extent
at start-time.</p>

<a name="defaultalloc"></a>

<h4>3.1.1 The default memory allocator</h4>

<p>By default, SQLite uses the malloc(), realloc(), and free() routines
from the standard C library for its memory allocation needs.  These routines
are surrounded by a thin wrapper that also provides a "memsize()" function
that will return the size of an existing allocation.  The memsize() function
is needed to keep an accurate count of the number of bytes of outstanding
memory; memsize() determines how many bytes to remove from the outstanding
count when an allocation is freed.  The default allocator implements
memsize() by always allocating 8 extra bytes on each malloc() request and
storing the size of the allocation in that 8-byte header.</p>

<p>The default memory allocator is recommended for most applications.
If you do not have a compelling need to use an alternative memory
allocator, then use the default.</p>

<a name="memdebug"></a>

<h4>3.1.2 The debugging memory allocator</h4>

<p>If SQLite is compiled with the <a href="compile.html#memdebug">SQLITE_MEMDEBUG</a> compile-time option,
then a different, heavy wrapper is used around system malloc(), realloc(), 
and free().
The heavy wrapper allocates around 100 bytes of extra space
with each allocation.  The extra space is used to places sentinel values 
at both ends of the allocation returned to the SQLite core.  When an
allocation is freed,
these sentinels are checked to make sure the SQLite core did not overrun
the buffer in either direction.  When the system library is GLIBC, the 
heavy wrapper also makes use of the GNU backtrace() function to examine
the stack and record the ancestor functions of the malloc() call.  When
running the SQLite test suite, the heavy wrapper also records the name of
the current test case.  These latter two features are useful for
tracking down the source of memory leaks detected by the test suite.</p>

<p>The heavy wrapper that is used when <a href="compile.html#memdebug">SQLITE_MEMDEBUG</a> is set also
makes sure each new allocation is filled with nonsense data prior to
returning the allocation to the caller.  And as soon as an allocation
is free, it is again filled with nonsense data.  These two actions help
to insure that the SQLite core does not make assumptions about the state
of newly allocated memory and that memory allocations are not used after
they have been freed.</p>

<p>The heavy wrapper employed by <a href="compile.html#memdebug">SQLITE_MEMDEBUG</a> is intended for use
only during testing, analysis, and debugging of SQLite.  The heavy wrapper
has a significant performance and memory overhead and probably should not
be used in production.</p>

<a name="memsys5"></a>

<h4>3.1.3 Zero-malloc memory allocator</h4>

<p>When SQLite is compiled with the <a href="compile.html#enable_memsys5">SQLITE_ENABLE_MEMSYS5</a> option, an
alternative memory allocator that does not use malloc() is included in the
build.  The SQLite developers refer to this alternative memory allocator
as "memsys5".  Even when it is included in the build, memsys5 is 
disabled by default.
To enable memsys5, the application must invoke the following SQLite 
interface at start-time:</p>

<blockquote><pre>
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a>, pBuf, szBuf, mnReq);
</pre></blockquote>

<p>In the call above, pBuf is a pointer to a large, contiguous chunk
of memory space that SQLite will use to satisfy all of its memory
allocation needs.   pBuf might point to a static array or it might
be memory obtained from some other application-specific mechanism.
szBuf is an integer that is the number of bytes of memory space
pointed to by pBuf.  mnReq is another integer that is the
minimum size of an allocation.  Any call to <a href="c3ref/free.html">sqlite3_malloc(N)</a> where
N is less than mnReq will be rounded up to mnReq.  mnReq must be
a power of two.  We shall see later that the mnReq parameter is
important in reducing the value of <b>n</b> and hence the minimum memory
size requirement in the <a href="malloc.html#nofrag">Robson proof</a>.</p>

<p>The memsys5 allocator is designed for use on embedded systems, 
though there is nothing to prevent its use on workstations.
The szBuf is typically between a few hundred kilobytes up to a few
dozen megabytes, depending on system requirements and memory budget.</p>

<p>The algorithm used by memsys5 can be called "power-of-two,
first-fit".  The sizes of all memory allocation 
requests are rounded up to a power of two and the request is satisfied
by the first free slot in pBuf that is large enough.  Adjacent freed
allocations are coalesced using a buddy system. When used appropriately,
this algorithm provides mathematical guarantees against fragmentation and
breakdown, as described further <a href="#nofrag">below</a>.</p>

<a name="memsysx"></a>

<h4>3.1.4 Experimental memory allocators</h4>

<p>The name "memsys5" used for the zero-malloc memory allocator implies
that there are several additional memory allocators available, and indeed
there are.  The default memory allocator is "memsys1".  The debugging
memory allocator is "memsys2".  Those have already been covered.</p>

<p>If SQLite is compiled with <a href="compile.html#enable_memsys3">SQLITE_ENABLE_MEMSYS3</a> than another
zero-malloc memory allocator, similar to memsys5, is included in the
source tree.  The memsys3 allocator, like memsys5, must be activated
by a call to <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a>,...).  Memsys3
uses the memory buffer supplied as its source for all memory allocations.
The difference between memsys3 and memsys5 is that memsys3 uses a
different memory allocation algorithm that seems to work well in
practice, but which does not provide mathematical
guarantees against memory fragmentation and breakdown.  Memsys3 was
a predecessor to memsys5.  The SQLite developers now believe that 
memsys5 is superior to
memsys3 and that all applications that need a zero-malloc memory
allocator should use memsys5 in preference to memsys3.  Memsys3 is
considered both experimental and deprecated and will likely be removed 
from the source tree in a future release of SQLite.</p>

<p>Code for memsys4 is still in the SQLite source tree (as of this writing - 
SQLite <a href="releaselog/3_6_1.html">version 3.6.1</a>), but it has not been maintained for several release
cycles and probably does not work.  (Update: memsys4 was removed as
of <a href="releaselog/3_6_5.html">version 3.6.5</a>) Memsys4 was an attempt to use mmap()
to obtain memory and then use madvise() to release unused pages
back to the operating system so that they could be reused by other
processes.  The work on memsys4 has been abandoned and the memsys4 module
will likely be removed from the source tree in the near future.</p>

<p>Memsys6 uses system malloc() and free() to obtain the memory it needs.
Memsys6 serves as an aggregator.  Memsys6 only calls system malloc() to obtain
large allocations.  It then subdivides those large allocations to services 
multiple smaller memory allocation requests coming from the SQLite core.
Memsys6 is intended for use in systems where
system malloc() is particularly inefficient.  The idea behind memsys6 is
to reduce the number of calls to system malloc() by a factor of 10 or more.</p>

<p>Memsys6 is made available by compiling SQLite with the SQLITE_ENABLE_MEMSYS6
compile-time option and then at start-time invoking:</p>

<blockquote><pre>
<a href="c3ref/config.html">sqlite3_config</a>(SQLITE_CONFIG_CHUNKALLOC);
</pre></blockquote>

<p>Memsys6 was added in SQLite <a href="releaselog/3_6_1.html">version 3.6.1</a>.
It is very experimental.  Its future is uncertain and it may be removed
in a subsequent release.  Update:  Memsys6 was removed as of 
<a href="releaselog/3_6_5.html">version 3.6.5</a>.</p>

<p>Other experimental memory allocators might be added in future releases
of SQLite.  One many anticipate that these will be called memsys7, memsys8,
and so forth.</p>

<a name="appalloc"></a>
<h4>3.1.5 Application-defined memory allocators</h4>

<p>New memory allocators do not have to be part of the SQLite source tree
nor included in the sqlite3.c <a href="amalgamation.html">amalgamation</a>.  Individual applications can
supply their own memory allocators to SQLite at start-time.</p>

<p>To cause SQLite to use a new memory allocator, the application
simply calls:</p>

<blockquote><pre>
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a>, pMem);
</pre></blockquote>

<p>In the call above, pMem is a pointer to an <a href="c3ref/mem_methods.html">sqlite3_mem_methods</a> object
that defines the interface to the application-specific memory allocator.
The <a href="c3ref/mem_methods.html">sqlite3_mem_methods</a> object is really just a structure containing
pointers to functions to implement the various memory allocation primitives.
</p>

<p>In a multi-threaded application, access to the <a href="c3ref/mem_methods.html">sqlite3_mem_methods</a>
is serialized if and only if <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a> is enabled.
If <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a> is disabled then the methods in
<a href="c3ref/mem_methods.html">sqlite3_mem_methods</a> must take care of their own serialization needs.</p>

<a name="overlayalloc"></a>
<h4>3.1.6 Memory allocator overlays</h4>

<p>An application can insert layers or "overlays" in between the
SQLite core and the underlying memory allocator.
For example, the <a href="#oomtesting">out-of-memory test logic</a>
for SQLite uses an overlay that can simulate memory allocation
failures.</p>

<p>An overlay can be created by using the</p>

<blockquote><pre>
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_GETMALLOC</a>, pOldMem);
</pre></blockquote>

<p>interface to obtain pointers to the existing memory allocator.
The existing allocator is saved by the overlay and is used as
a fallback to do real memory allocation.  Then the overlay is
inserted in place of the existing memory allocator using
the <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a>,...) as described
<a href="#appalloc">above</a>.

<a name="stuballoc"></a>
<h4>3.1.7 No-op memory allocator stub</h4>

<p>If SQLite is compiled with the <a href="compile.html#zero_malloc">SQLITE_ZERO_MALLOC</a> option, then
the <a href="malloc.html#defaultalloc">default memory allocator</a> is omitted and replaced by a stub
memory allocator that never allocates any memory.  Any calls to the
stub memory allocator will report back that no memory is available.</p>

<p>The no-op memory allocator is not useful by itself.  It exists only
as a placeholder so that SQLite has a memory allocator to link against
on systems that may not have malloc(), free(), or realloc() in their
standard library.
An application that is compiled with <a href="compile.html#zero_malloc">SQLITE_ZERO_MALLOC</a> will need to
use <a href="c3ref/config.html">sqlite3_config()</a> together with <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a> or
<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a> to specify a new alternative memory allocator
before beginning to use SQLite.</p>

<a name="scratch"></a>

<h3>3.2 Scratch memory</h3>

<p>SQLite occasionally needs a large chunk of "scratch" memory to
perform some transient calculation.  Scratch memory is used, for example,
as temporary storage when rebalancing a B-Tree.  These scratch memory
allocations are typically about 10 kilobytes in size and are
transient - lasting
only for the duration of a single, short-lived function call.</p>

<p>In older versions of SQLite, the scratch memory was obtained from
the processor stack.  That works great on workstations with a large stack.
But pulling large buffers from the stack 
caused problems on embedded systems with a 
small processor stack (typically 4K or 8K).  And so SQLite was modified
to allocate scratch memory from the heap.</p>

<p>However, doing occasional large transient allocations from the heap can
lead to memory fragmentation in embedded systems.  To work around this
problem, a separate memory allocation system for scratch memory has been
created.</p>

<p>The scratch memory allocator is set up as follows:</p>

<blockquote><pre>
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_SCRATCH</a>, pBuf, sz, N);
</pre></blockquote>