manual.xml.svn-base

絲路server源碼 Silk Road server source

their help, support and advice.  See the manual in the source
distribution for pointers to sources of documentation.  Christian
von Roques encouraged me to look for faster sorting algorithms,
so as to speed up compression.  Bela Lubkin encouraged me to
improve the worst-case compression performance.  
Donna Robinson XMLised the documentation.
Many people sent
patches, helped with portability problems, lent machines, gave
advice and were generally helpful.</para>

</sect1>

</chapter>



<chapter id="libprog" xreflabel="Programming with libbzip2">
<title>
Programming with <computeroutput>libbzip2</computeroutput>
</title>

<para>This chapter describes the programming interface to
<computeroutput>libbzip2</computeroutput>.</para>

<para>For general background information, particularly about
memory use and performance aspects, you'd be well advised to read
<xref linkend="using"/> as well.</para>


<sect1 id="top-level" xreflabel="Top-level structure">
<title>Top-level structure</title>

<para><computeroutput>libbzip2</computeroutput> is a flexible
library for compressing and decompressing data in the
<computeroutput>bzip2</computeroutput> data format.  Although
packaged as a single entity, it helps to regard the library as
three separate parts: the low level interface, and the high level
interface, and some utility functions.</para>

<para>The structure of
<computeroutput>libbzip2</computeroutput>'s interfaces is similar
to that of Jean-loup Gailly's and Mark Adler's excellent
<computeroutput>zlib</computeroutput> library.</para>

<para>All externally visible symbols have names beginning
<computeroutput>BZ2_</computeroutput>.  This is new in version
1.0.  The intention is to minimise pollution of the namespaces of
library clients.</para>

<para>To use any part of the library, you need to
<computeroutput>#include &lt;bzlib.h&gt;</computeroutput>
into your sources.</para>



<sect2 id="ll-summary" xreflabel="Low-level summary">
<title>Low-level summary</title>

<para>This interface provides services for compressing and
decompressing data in memory.  There's no provision for dealing
with files, streams or any other I/O mechanisms, just straight
memory-to-memory work.  In fact, this part of the library can be
compiled without inclusion of
<computeroutput>stdio.h</computeroutput>, which may be helpful
for embedded applications.</para>

<para>The low-level part of the library has no global variables
and is therefore thread-safe.</para>

<para>Six routines make up the low level interface:
<computeroutput>BZ2_bzCompressInit</computeroutput>,
<computeroutput>BZ2_bzCompress</computeroutput>, and
<computeroutput>BZ2_bzCompressEnd</computeroutput> for
compression, and a corresponding trio
<computeroutput>BZ2_bzDecompressInit</computeroutput>,
<computeroutput>BZ2_bzDecompress</computeroutput> and
<computeroutput>BZ2_bzDecompressEnd</computeroutput> for
decompression.  The <computeroutput>*Init</computeroutput>
functions allocate memory for compression/decompression and do
other initialisations, whilst the
<computeroutput>*End</computeroutput> functions close down
operations and release memory.</para>

<para>The real work is done by
<computeroutput>BZ2_bzCompress</computeroutput> and
<computeroutput>BZ2_bzDecompress</computeroutput>.  These
compress and decompress data from a user-supplied input buffer to
a user-supplied output buffer.  These buffers can be any size;
arbitrary quantities of data are handled by making repeated calls
to these functions.  This is a flexible mechanism allowing a
consumer-pull style of activity, or producer-push, or a mixture
of both.</para>

</sect2>


<sect2 id="hl-summary" xreflabel="High-level summary">
<title>High-level summary</title>

<para>This interface provides some handy wrappers around the
low-level interface to facilitate reading and writing
<computeroutput>bzip2</computeroutput> format files
(<computeroutput>.bz2</computeroutput> files).  The routines
provide hooks to facilitate reading files in which the
<computeroutput>bzip2</computeroutput> data stream is embedded
within some larger-scale file structure, or where there are
multiple <computeroutput>bzip2</computeroutput> data streams
concatenated end-to-end.</para>

<para>For reading files,
<computeroutput>BZ2_bzReadOpen</computeroutput>,
<computeroutput>BZ2_bzRead</computeroutput>,
<computeroutput>BZ2_bzReadClose</computeroutput> and 
<computeroutput>BZ2_bzReadGetUnused</computeroutput> are
supplied.  For writing files,
<computeroutput>BZ2_bzWriteOpen</computeroutput>,
<computeroutput>BZ2_bzWrite</computeroutput> and
<computeroutput>BZ2_bzWriteFinish</computeroutput> are
available.</para>

<para>As with the low-level library, no global variables are used
so the library is per se thread-safe.  However, if I/O errors
occur whilst reading or writing the underlying compressed files,
you may have to consult <computeroutput>errno</computeroutput> to
determine the cause of the error.  In that case, you'd need a C
library which correctly supports
<computeroutput>errno</computeroutput> in a multithreaded
environment.</para>

<para>To make the library a little simpler and more portable,
<computeroutput>BZ2_bzReadOpen</computeroutput> and
<computeroutput>BZ2_bzWriteOpen</computeroutput> require you to
pass them file handles (<computeroutput>FILE*</computeroutput>s)
which have previously been opened for reading or writing
respectively.  That avoids portability problems associated with
file operations and file attributes, whilst not being much of an
imposition on the programmer.</para>

</sect2>


<sect2 id="util-fns-summary" xreflabel="Utility functions summary">
<title>Utility functions summary</title>

<para>For very simple needs,
<computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and
<computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> are
provided.  These compress data in memory from one buffer to
another buffer in a single function call.  You should assess
whether these functions fulfill your memory-to-memory
compression/decompression requirements before investing effort in
understanding the more general but more complex low-level
interface.</para>

<para>Yoshioka Tsuneo
(<computeroutput>[e-mail]</computeroutput> /
<computeroutput>[e-mail]</computeroutput>) has
contributed some functions to give better
<computeroutput>zlib</computeroutput> compatibility.  These
functions are <computeroutput>BZ2_bzopen</computeroutput>,
<computeroutput>BZ2_bzread</computeroutput>,
<computeroutput>BZ2_bzwrite</computeroutput>,
<computeroutput>BZ2_bzflush</computeroutput>,
<computeroutput>BZ2_bzclose</computeroutput>,
<computeroutput>BZ2_bzerror</computeroutput> and
<computeroutput>BZ2_bzlibVersion</computeroutput>.  You may find
these functions more convenient for simple file reading and
writing, than those in the high-level interface.  These functions
are not (yet) officially part of the library, and are minimally
documented here.  If they break, you get to keep all the pieces.
I hope to document them properly when time permits.</para>

<para>Yoshioka also contributed modifications to allow the
library to be built as a Windows DLL.</para>

</sect2>

</sect1>


<sect1 id="err-handling" xreflabel="Error handling">
<title>Error handling</title>

<para>The library is designed to recover cleanly in all
situations, including the worst-case situation of decompressing
random data.  I'm not 100% sure that it can always do this, so
you might want to add a signal handler to catch segmentation
violations during decompression if you are feeling especially
paranoid.  I would be interested in hearing more about the
robustness of the library to corrupted compressed data.</para>

<para>Version 1.0.3 more robust in this respect than any
previous version.  Investigations with Valgrind (a tool for detecting
problems with memory management) indicate
that, at least for the few files I tested, all single-bit errors
in the decompressed data are caught properly, with no
segmentation faults, no uses of uninitialised data, no out of
range reads or writes, and no infinite looping in the decompressor.
So it's certainly pretty robust, although
I wouldn't claim it to be totally bombproof.</para>

<para>The file <computeroutput>bzlib.h</computeroutput> contains
all definitions needed to use the library.  In particular, you
should definitely not include
<computeroutput>bzlib_private.h</computeroutput>.</para>

<para>In <computeroutput>bzlib.h</computeroutput>, the various
return values are defined.  The following list is not intended as
an exhaustive description of the circumstances in which a given
value may be returned -- those descriptions are given later.
Rather, it is intended to convey the rough meaning of each return
value.  The first five actions are normal and not intended to
denote an error situation.</para>

<variablelist>

 <varlistentry>
  <term><computeroutput>BZ_OK</computeroutput></term>
  <listitem><para>The requested action was completed
   successfully.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_RUN_OK, BZ_FLUSH_OK,
    BZ_FINISH_OK</computeroutput></term>
  <listitem><para>In 
   <computeroutput>BZ2_bzCompress</computeroutput>, the requested
   flush/finish/nothing-special action was completed
   successfully.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_STREAM_END</computeroutput></term>
  <listitem><para>Compression of data was completed, or the
   logical stream end was detected during
   decompression.</para></listitem>
 </varlistentry>

</variablelist>

<para>The following return values indicate an error of some
kind.</para>

<variablelist>

 <varlistentry>
  <term><computeroutput>BZ_CONFIG_ERROR</computeroutput></term>
  <listitem><para>Indicates that the library has been improperly
   compiled on your platform -- a major configuration error.
   Specifically, it means that
   <computeroutput>sizeof(char)</computeroutput>,
   <computeroutput>sizeof(short)</computeroutput> and
   <computeroutput>sizeof(int)</computeroutput> are not 1, 2 and
   4 respectively, as they should be.  Note that the library
   should still work properly on 64-bit platforms which follow
   the LP64 programming model -- that is, where
   <computeroutput>sizeof(long)</computeroutput> and
   <computeroutput>sizeof(void*)</computeroutput> are 8.  Under
   LP64, <computeroutput>sizeof(int)</computeroutput> is still 4,
   so <computeroutput>libbzip2</computeroutput>, which doesn't
   use the <computeroutput>long</computeroutput> type, is
   OK.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_SEQUENCE_ERROR</computeroutput></term>
  <listitem><para>When using the library, it is important to call
   the functions in the correct sequence and with data structures
   (buffers etc) in the correct states.
   <computeroutput>libbzip2</computeroutput> checks as much as it
   can to ensure this is happening, and returns
   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> if not.
   Code which complies precisely with the function semantics, as
   detailed below, should never receive this value; such an event
   denotes buggy code which you should
   investigate.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_PARAM_ERROR</computeroutput></term>
  <listitem><para>Returned when a parameter to a function call is
   out of range or otherwise manifestly incorrect.  As with
   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput>, this
   denotes a bug in the client code.  The distinction between
   <computeroutput>BZ_PARAM_ERROR</computeroutput> and
   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> is a bit
   hazy, but still worth making.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_MEM_ERROR</computeroutput></term>
  <listitem><para>Returned when a request to allocate memory
   failed.  Note that the quantity of memory needed to decompress
   a stream cannot be determined until the stream's header has
   been read.  So
   <computeroutput>BZ2_bzDecompress</computeroutput> and
   <computeroutput>BZ2_bzRead</computeroutput> may return
   <computeroutput>BZ_MEM_ERROR</computeroutput> even though some
   of the compressed data has been read.  The same is not true
   for compression; once
   <computeroutput>BZ2_bzCompressInit</computeroutput> or
   <computeroutput>BZ2_bzWriteOpen</computeroutput> have
   successfully completed,
   <computeroutput>BZ_MEM_ERROR</computeroutput> cannot
   occur.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_DATA_ERROR</computeroutput></term>
  <listitem><para>Returned when a data integrity error is
   detected during decompression.  Most importantly, this means
   when stored and computed CRCs for the data do not match.  This
   value is also returned upon detection of any other anomaly in
   the compressed data.</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_DATA_ERROR_MAGIC</computeroutput></term>
  <listitem><para>As a special case of
   <computeroutput>BZ_DATA_ERROR</computeroutput>, it is
   sometimes useful to know when the compressed stream does not
   start with the correct magic bytes (<computeroutput>'B' 'Z'
   'h'</computeroutput>).</para></listitem>
 </varlistentry>

 <varlistentry>
  <term><computeroutput>BZ_IO_ERROR</computeroutput></term>
  <listitem><para>Returned by
   <computeroutput>BZ2_bzRead</computeroutput> and
   <computeroutput>BZ2_bzWrite</computeroutput> when there is an
   error reading or writing in the compressed file, and by
   <computeroutput>BZ2_bzReadOpen</computeroutput> and
   <computeroutput>BZ2_bzWriteOpen</computeroutput> for attempts