manual.xml.svn-base

絲路server源碼 Silk Road server source

   to use a file for which the error indicator (viz,
   <computeroutput>ferror(f)</computeroutput>) is set.  On
   receipt of <computeroutput>BZ_IO_ERROR</computeroutput>, the
   caller should consult <computeroutput>errno</computeroutput>
   and/or <computeroutput>perror</computeroutput> to acquire
   operating-system specific information about the
   problem.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_UNEXPECTED_EOF</computeroutput></term>
  <listitem><para>Returned by
   <computeroutput>BZ2_bzRead</computeroutput> when the
   compressed file finishes before the logical end of stream is
   detected.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_OUTBUFF_FULL</computeroutput></term>
  <listitem><para>Returned by
   <computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and
   <computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> to
   indicate that the output data will not fit into the output
   buffer provided.</para></listitem>
 </varlistentry>

</variablelist>

</sect1>



<sect1 id="low-level" xreflabel=">Low-level interface">
<title>Low-level interface</title>


<sect2 id="bzcompress-init" xreflabel="BZ2_bzCompressInit">
<title><computeroutput>BZ2_bzCompressInit</computeroutput></title>

<programlisting>
typedef struct {
  char *next_in;
  unsigned int avail_in;
  unsigned int total_in_lo32;
  unsigned int total_in_hi32;

  char *next_out;
  unsigned int avail_out;
  unsigned int total_out_lo32;
  unsigned int total_out_hi32;

  void *state;

  void *(*bzalloc)(void *,int,int);
  void (*bzfree)(void *,void *);
  void *opaque;
} bz_stream;

int BZ2_bzCompressInit ( bz_stream *strm, 
                         int blockSize100k, 
                         int verbosity,
                         int workFactor );
</programlisting>

<para>Prepares for compression.  The
<computeroutput>bz_stream</computeroutput> structure holds all
data pertaining to the compression activity.  A
<computeroutput>bz_stream</computeroutput> structure should be
allocated and initialised prior to the call.  The fields of
<computeroutput>bz_stream</computeroutput> comprise the entirety
of the user-visible data.  <computeroutput>state</computeroutput>
is a pointer to the private data structures required for
compression.</para>

<para>Custom memory allocators are supported, via fields
<computeroutput>bzalloc</computeroutput>,
<computeroutput>bzfree</computeroutput>, and
<computeroutput>opaque</computeroutput>.  The value
<computeroutput>opaque</computeroutput> is passed to as the first
argument to all calls to <computeroutput>bzalloc</computeroutput>
and <computeroutput>bzfree</computeroutput>, but is otherwise
ignored by the library.  The call <computeroutput>bzalloc (
opaque, n, m )</computeroutput> is expected to return a pointer
<computeroutput>p</computeroutput> to <computeroutput>n *
m</computeroutput> bytes of memory, and <computeroutput>bzfree (
opaque, p )</computeroutput> should free that memory.</para>

<para>If you don't want to use a custom memory allocator, set
<computeroutput>bzalloc</computeroutput>,
<computeroutput>bzfree</computeroutput> and
<computeroutput>opaque</computeroutput> to
<computeroutput>NULL</computeroutput>, and the library will then
use the standard <computeroutput>malloc</computeroutput> /
<computeroutput>free</computeroutput> routines.</para>

<para>Before calling
<computeroutput>BZ2_bzCompressInit</computeroutput>, fields
<computeroutput>bzalloc</computeroutput>,
<computeroutput>bzfree</computeroutput> and
<computeroutput>opaque</computeroutput> should be filled
appropriately, as just described.  Upon return, the internal
state will have been allocated and initialised, and
<computeroutput>total_in_lo32</computeroutput>,
<computeroutput>total_in_hi32</computeroutput>,
<computeroutput>total_out_lo32</computeroutput> and
<computeroutput>total_out_hi32</computeroutput> will have been
set to zero.  These four fields are used by the library to inform
the caller of the total amount of data passed into and out of the
library, respectively.  You should not try to change them.  As of
version 1.0, 64-bit counts are maintained, even on 32-bit
platforms, using the <computeroutput>_hi32</computeroutput>
fields to store the upper 32 bits of the count.  So, for example,
the total amount of data in is <computeroutput>(total_in_hi32
&#60;&#60; 32) + total_in_lo32</computeroutput>.</para>

<para>Parameter <computeroutput>blockSize100k</computeroutput>
specifies the block size to be used for compression.  It should
be a value between 1 and 9 inclusive, and the actual block size
used is 100000 x this figure.  9 gives the best compression but
takes most memory.</para>

<para>Parameter <computeroutput>verbosity</computeroutput> should
be set to a number between 0 and 4 inclusive.  0 is silent, and
greater numbers give increasingly verbose monitoring/debugging
output.  If the library has been compiled with
<computeroutput>-DBZ_NO_STDIO</computeroutput>, no such output
will appear for any verbosity setting.</para>

<para>Parameter <computeroutput>workFactor</computeroutput>
controls how the compression phase behaves when presented with
worst case, highly repetitive, input data.  If compression runs
into difficulties caused by repetitive data, the library switches
from the standard sorting algorithm to a fallback algorithm.  The
fallback is slower than the standard algorithm by perhaps a
factor of three, but always behaves reasonably, no matter how bad
the input.</para>

<para>Lower values of <computeroutput>workFactor</computeroutput>
reduce the amount of effort the standard algorithm will expend
before resorting to the fallback.  You should set this parameter
carefully; too low, and many inputs will be handled by the
fallback algorithm and so compress rather slowly, too high, and
your average-to-worst case compression times can become very
large.  The default value of 30 gives reasonable behaviour over a
wide range of circumstances.</para>

<para>Allowable values range from 0 to 250 inclusive.  0 is a
special case, equivalent to using the default value of 30.</para>

<para>Note that the compressed output generated is the same
regardless of whether or not the fallback algorithm is
used.</para>

<para>Be aware also that this parameter may disappear entirely in
future versions of the library.  In principle it should be
possible to devise a good way to automatically choose which
algorithm to use.  Such a mechanism would render the parameter
obsolete.</para>

<para>Possible return values:</para>

<programlisting>
BZ_CONFIG_ERROR
  if the library has been mis-compiled
BZ_PARAM_ERROR
  if strm is NULL 
  or blockSize < 1 or blockSize > 9
  or verbosity < 0 or verbosity > 4
  or workFactor < 0 or workFactor > 250
BZ_MEM_ERROR 
  if not enough memory is available
BZ_OK 
  otherwise
</programlisting>

<para>Allowable next actions:</para>

<programlisting>
BZ2_bzCompress
  if BZ_OK is returned
  no specific action needed in case of error
</programlisting>

</sect2>


<sect2 id="bzCompress" xreflabel="BZ2_bzCompress">
<title><computeroutput>BZ2_bzCompress</computeroutput></title>

<programlisting>
int BZ2_bzCompress ( bz_stream *strm, int action );
</programlisting>

<para>Provides more input and/or output buffer space for the
library.  The caller maintains input and output buffers, and
calls <computeroutput>BZ2_bzCompress</computeroutput> to transfer
data between them.</para>

<para>Before each call to
<computeroutput>BZ2_bzCompress</computeroutput>,
<computeroutput>next_in</computeroutput> should point at the data
to be compressed, and <computeroutput>avail_in</computeroutput>
should indicate how many bytes the library may read.
<computeroutput>BZ2_bzCompress</computeroutput> updates
<computeroutput>next_in</computeroutput>,
<computeroutput>avail_in</computeroutput> and
<computeroutput>total_in</computeroutput> to reflect the number
of bytes it has read.</para>

<para>Similarly, <computeroutput>next_out</computeroutput> should
point to a buffer in which the compressed data is to be placed,
with <computeroutput>avail_out</computeroutput> indicating how
much output space is available.
<computeroutput>BZ2_bzCompress</computeroutput> updates
<computeroutput>next_out</computeroutput>,
<computeroutput>avail_out</computeroutput> and
<computeroutput>total_out</computeroutput> to reflect the number
of bytes output.</para>

<para>You may provide and remove as little or as much data as you
like on each call of
<computeroutput>BZ2_bzCompress</computeroutput>.  In the limit,
it is acceptable to supply and remove data one byte at a time,
although this would be terribly inefficient.  You should always
ensure that at least one byte of output space is available at
each call.</para>

<para>A second purpose of
<computeroutput>BZ2_bzCompress</computeroutput> is to request a
change of mode of the compressed stream.</para>

<para>Conceptually, a compressed stream can be in one of four
states: IDLE, RUNNING, FLUSHING and FINISHING.  Before
initialisation
(<computeroutput>BZ2_bzCompressInit</computeroutput>) and after
termination (<computeroutput>BZ2_bzCompressEnd</computeroutput>),
a stream is regarded as IDLE.</para>

<para>Upon initialisation
(<computeroutput>BZ2_bzCompressInit</computeroutput>), the stream
is placed in the RUNNING state.  Subsequent calls to
<computeroutput>BZ2_bzCompress</computeroutput> should pass
<computeroutput>BZ_RUN</computeroutput> as the requested action;
other actions are illegal and will result in
<computeroutput>BZ_SEQUENCE_ERROR</computeroutput>.</para>

<para>At some point, the calling program will have provided all
the input data it wants to.  It will then want to finish up -- in
effect, asking the library to process any data it might have
buffered internally.  In this state,
<computeroutput>BZ2_bzCompress</computeroutput> will no longer
attempt to read data from
<computeroutput>next_in</computeroutput>, but it will want to
write data to <computeroutput>next_out</computeroutput>.  Because
the output buffer supplied by the user can be arbitrarily small,
the finishing-up operation cannot necessarily be done with a
single call of
<computeroutput>BZ2_bzCompress</computeroutput>.</para>

<para>Instead, the calling program passes
<computeroutput>BZ_FINISH</computeroutput> as an action to
<computeroutput>BZ2_bzCompress</computeroutput>.  This changes
the stream's state to FINISHING.  Any remaining input (ie,
<computeroutput>next_in[0 .. avail_in-1]</computeroutput>) is
compressed and transferred to the output buffer.  To do this,
<computeroutput>BZ2_bzCompress</computeroutput> must be called
repeatedly until all the output has been consumed.  At that
point, <computeroutput>BZ2_bzCompress</computeroutput> returns
<computeroutput>BZ_STREAM_END</computeroutput>, and the stream's
state is set back to IDLE.
<computeroutput>BZ2_bzCompressEnd</computeroutput> should then be
called.</para>

<para>Just to make sure the calling program does not cheat, the
library makes a note of <computeroutput>avail_in</computeroutput>
at the time of the first call to
<computeroutput>BZ2_bzCompress</computeroutput> which has
<computeroutput>BZ_FINISH</computeroutput> as an action (ie, at
the time the program has announced its intention to not supply
any more input).  By comparing this value with that of
<computeroutput>avail_in</computeroutput> over subsequent calls
to <computeroutput>BZ2_bzCompress</computeroutput>, the library
can detect any attempts to slip in more data to compress.  Any
calls for which this is detected will return
<computeroutput>BZ_SEQUENCE_ERROR</computeroutput>.  This
indicates a programming mistake which should be corrected.</para>

<para>Instead of asking to finish, the calling program may ask
<computeroutput>BZ2_bzCompress</computeroutput> to take all the
remaining input, compress it and terminate the current
(Burrows-Wheeler) compression block.  This could be useful for
error control purposes.  The mechanism is analogous to that for
finishing: call <computeroutput>BZ2_bzCompress</computeroutput>
with an action of <computeroutput>BZ_FLUSH</computeroutput>,
remove output data, and persist with the
<computeroutput>BZ_FLUSH</computeroutput> action until the value
<computeroutput>BZ_RUN</computeroutput> is returned.  As with
finishing, <computeroutput>BZ2_bzCompress</computeroutput>
detects any attempt to provide more input data once the flush has
begun.</para>

<para>Once the flush is complete, the stream returns to the
normal RUNNING state.</para>

<para>This all sounds pretty complex, but isn't really.  Here's a
table which shows which actions are allowable in each state, what
action will be taken, what the next state is, and what the
non-error return values are.  Note that you can't explicitly ask
what state the stream is in, but nor do you need to -- it can be
inferred from the values returned by
<computeroutput>BZ2_bzCompress</computeroutput>.</para>

<programlisting>
IDLE/any
  Illegal.  IDLE state only exists after BZ2_bzCompressEnd or
  before BZ2_bzCompressInit.
  Return value = BZ_SEQUENCE_ERROR

RUNNING/BZ_RUN
  Compress from next_in to next_out as much as possible.
  Next state = RUNNING
  Return value = BZ_RUN_OK

RUNNING/BZ_FLUSH
  Remember current value of next_in. Compress from next_in
  to next_out as much as possible, but do not accept any more input.
  Next state = FLUSHING
  Return value = BZ_FLUSH_OK

RUNNING/BZ_FINISH
  Remember current value of next_in. Compress from next_in
  to next_out as much as possible, but do not accept any more input.
  Next state = FINISHING