manual.xml.svn-base

絲路server源碼 Silk Road server source

 <listitem><para>Force decompression.
  <computeroutput>bzip2</computeroutput>,
  <computeroutput>bunzip2</computeroutput> and
  <computeroutput>bzcat</computeroutput> are really the same
  program, and the decision about what actions to take is done on
  the basis of which name is used.  This flag overrides that
  mechanism, and forces bzip2 to decompress.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-z --compress</computeroutput></term>
 <listitem><para>The complement to
  <computeroutput>-d</computeroutput>: forces compression,
  regardless of the invokation name.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-t --test</computeroutput></term>
 <listitem><para>Check integrity of the specified file(s), but
  don't decompress them.  This really performs a trial
  decompression and throws away the result.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-f --force</computeroutput></term>
 <listitem><para>Force overwrite of output files.  Normally,
  <computeroutput>bzip2</computeroutput> will not overwrite
  existing output files.  Also forces
  <computeroutput>bzip2</computeroutput> to break hard links to
  files, which it otherwise wouldn't do.</para>
  <para><computeroutput>bzip2</computeroutput> normally declines
  to decompress files which don't have the correct magic header
  bytes. If forced (<computeroutput>-f</computeroutput>),
  however, it will pass such files through unmodified. This is
  how GNU <computeroutput>gzip</computeroutput> behaves.</para>
 </listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-k --keep</computeroutput></term>
 <listitem><para>Keep (don't delete) input files during
  compression or decompression.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-s --small</computeroutput></term>
 <listitem><para>Reduce memory usage, for compression,
  decompression and testing.  Files are decompressed and tested
  using a modified algorithm which only requires 2.5 bytes per
  block byte.  This means any file can be decompressed in 2300k
  of memory, albeit at about half the normal speed.</para>
  <para>During compression, <computeroutput>-s</computeroutput>
  selects a block size of 200k, which limits memory use to around
  the same figure, at the expense of your compression ratio.  In
  short, if your machine is low on memory (8 megabytes or less),
  use <computeroutput>-s</computeroutput> for everything.  See
  <xref linkend="memory-management"/> below.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-q --quiet</computeroutput></term>
 <listitem><para>Suppress non-essential warning messages.
  Messages pertaining to I/O errors and other critical events
  will not be suppressed.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-v --verbose</computeroutput></term>
 <listitem><para>Verbose mode -- show the compression ratio for
  each file processed.  Further
  <computeroutput>-v</computeroutput>'s increase the verbosity
  level, spewing out lots of information which is primarily of
  interest for diagnostic purposes.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-L --license -V --version</computeroutput></term>
 <listitem><para>Display the software version, license terms and
  conditions.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>-1</computeroutput> (or
 <computeroutput>--fast</computeroutput>) to
 <computeroutput>-9</computeroutput> (or
 <computeroutput>-best</computeroutput>)</term>
 <listitem><para>Set the block size to 100 k, 200 k ...  900 k
  when compressing.  Has no effect when decompressing.  See <xref
  linkend="memory-management" /> below.  The
  <computeroutput>--fast</computeroutput> and
  <computeroutput>--best</computeroutput> aliases are primarily
  for GNU <computeroutput>gzip</computeroutput> compatibility.
  In particular, <computeroutput>--fast</computeroutput> doesn't
  make things significantly faster.  And
  <computeroutput>--best</computeroutput> merely selects the
  default behaviour.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>--</computeroutput></term>
 <listitem><para>Treats all subsequent arguments as file names,
  even if they start with a dash.  This is so you can handle
  files with names beginning with a dash, for example:
  <computeroutput>bzip2 --
  -myfilename</computeroutput>.</para></listitem>
 </varlistentry>

 <varlistentry>
 <term><computeroutput>--repetitive-fast</computeroutput></term>
 <term><computeroutput>--repetitive-best</computeroutput></term>
 <listitem><para>These flags are redundant in versions 0.9.5 and
  above.  They provided some coarse control over the behaviour of
  the sorting algorithm in earlier versions, which was sometimes
  useful.  0.9.5 and above have an improved algorithm which
  renders these flags irrelevant.</para></listitem>
 </varlistentry>

</variablelist>

</sect1>


<sect1 id="memory-management" xreflabel="MEMORY MANAGEMENT">
<title>MEMORY MANAGEMENT</title>

<para><computeroutput>bzip2</computeroutput> compresses large
files in blocks.  The block size affects both the compression
ratio achieved, and the amount of memory needed for compression
and decompression.  The flags <computeroutput>-1</computeroutput>
through <computeroutput>-9</computeroutput> specify the block
size to be 100,000 bytes through 900,000 bytes (the default)
respectively.  At decompression time, the block size used for
compression is read from the header of the compressed file, and
<computeroutput>bunzip2</computeroutput> then allocates itself
just enough memory to decompress the file.  Since block sizes are
stored in compressed files, it follows that the flags
<computeroutput>-1</computeroutput> to
<computeroutput>-9</computeroutput> are irrelevant to and so
ignored during decompression.</para>

<para>Compression and decompression requirements, in bytes, can be
estimated as:</para>
<programlisting>
Compression:   400k + ( 8 x block size )

Decompression: 100k + ( 4 x block size ), or
               100k + ( 2.5 x block size )
</programlisting>

<para>Larger block sizes give rapidly diminishing marginal
returns.  Most of the compression comes from the first two or
three hundred k of block size, a fact worth bearing in mind when
using <computeroutput>bzip2</computeroutput> on small machines.
It is also important to appreciate that the decompression memory
requirement is set at compression time by the choice of block
size.</para>

<para>For files compressed with the default 900k block size,
<computeroutput>bunzip2</computeroutput> will require about 3700
kbytes to decompress.  To support decompression of any file on a
4 megabyte machine, <computeroutput>bunzip2</computeroutput> has
an option to decompress using approximately half this amount of
memory, about 2300 kbytes.  Decompression speed is also halved,
so you should use this option only where necessary.  The relevant
flag is <computeroutput>-s</computeroutput>.</para>

<para>In general, try and use the largest block size memory
constraints allow, since that maximises the compression achieved.
Compression and decompression speed are virtually unaffected by
block size.</para>

<para>Another significant point applies to files which fit in a
single block -- that means most files you'd encounter using a
large block size.  The amount of real memory touched is
proportional to the size of the file, since the file is smaller
than a block.  For example, compressing a file 20,000 bytes long
with the flag <computeroutput>-9</computeroutput> will cause the
compressor to allocate around 7600k of memory, but only touch
400k + 20000 * 8 = 560 kbytes of it.  Similarly, the decompressor
will allocate 3700k but only touch 100k + 20000 * 4 = 180
kbytes.</para>

<para>Here is a table which summarises the maximum memory usage
for different block sizes.  Also recorded is the total compressed
size for 14 files of the Calgary Text Compression Corpus
totalling 3,141,622 bytes.  This column gives some feel for how
compression varies with block size.  These figures tend to
understate the advantage of larger block sizes for larger files,
since the Corpus is dominated by smaller files.</para>

<programlisting>
        Compress   Decompress   Decompress   Corpus
Flag     usage      usage       -s usage     Size

 -1      1200k       500k         350k      914704
 -2      2000k       900k         600k      877703
 -3      2800k      1300k         850k      860338
 -4      3600k      1700k        1100k      846899
 -5      4400k      2100k        1350k      845160
 -6      5200k      2500k        1600k      838626
 -7      6100k      2900k        1850k      834096
 -8      6800k      3300k        2100k      828642
 -9      7600k      3700k        2350k      828642
</programlisting>

</sect1>


<sect1 id="recovering" xreflabel="RECOVERING DATA FROM DAMAGED FILES">
<title>RECOVERING DATA FROM DAMAGED FILES</title>

<para><computeroutput>bzip2</computeroutput> compresses files in
blocks, usually 900kbytes long.  Each block is handled
independently.  If a media or transmission error causes a
multi-block <computeroutput>.bz2</computeroutput> file to become
damaged, it may be possible to recover data from the undamaged
blocks in the file.</para>

<para>The compressed representation of each block is delimited by
a 48-bit pattern, which makes it possible to find the block
boundaries with reasonable certainty.  Each block also carries
its own 32-bit CRC, so damaged blocks can be distinguished from
undamaged ones.</para>

<para><computeroutput>bzip2recover</computeroutput> is a simple
program whose purpose is to search for blocks in
<computeroutput>.bz2</computeroutput> files, and write each block
out into its own <computeroutput>.bz2</computeroutput> file.  You
can then use <computeroutput>bzip2 -t</computeroutput> to test
the integrity of the resulting files, and decompress those which
are undamaged.</para>

<para><computeroutput>bzip2recover</computeroutput> takes a
single argument, the name of the damaged file, and writes a
number of files <computeroutput>rec0001file.bz2</computeroutput>,
<computeroutput>rec0002file.bz2</computeroutput>, etc, containing
the extracted blocks.  The output filenames are designed so that
the use of wildcards in subsequent processing -- for example,
<computeroutput>bzip2 -dc rec*file.bz2 &#62;
recovered_data</computeroutput> -- lists the files in the correct
order.</para>

<para><computeroutput>bzip2recover</computeroutput> should be of
most use dealing with large <computeroutput>.bz2</computeroutput>
files, as these will contain many blocks.  It is clearly futile
to use it on damaged single-block files, since a damaged block
cannot be recovered.  If you wish to minimise any potential data
loss through media or transmission errors, you might consider
compressing with a smaller block size.</para>

</sect1>


<sect1 id="performance" xreflabel="PERFORMANCE NOTES">
<title>PERFORMANCE NOTES</title>

<para>The sorting phase of compression gathers together similar
strings in the file.  Because of this, files containing very long
runs of repeated symbols, like "aabaabaabaab ..."  (repeated
several hundred times) may compress more slowly than normal.
Versions 0.9.5 and above fare much better than previous versions
in this respect.  The ratio between worst-case and average-case
compression time is in the region of 10:1.  For previous
versions, this figure was more like 100:1.  You can use the
<computeroutput>-vvvv</computeroutput> option to monitor progress
in great detail, if you want.</para>

<para>Decompression speed is unaffected by these
phenomena.</para>

<para><computeroutput>bzip2</computeroutput> usually allocates
several megabytes of memory to operate in, and then charges all
over it in a fairly random fashion.  This means that performance,
both for compressing and decompressing, is largely determined by
the speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the miss
rate have been observed to give disproportionately large
performance improvements.  I imagine
<computeroutput>bzip2</computeroutput> will perform best on
machines with very large caches.</para>

</sect1>



<sect1 id="caveats" xreflabel="CAVEATS">
<title>CAVEATS</title>

<para>I/O error messages are not as helpful as they could be.
<computeroutput>bzip2</computeroutput> tries hard to detect I/O
errors and exit cleanly, but the details of what the problem is
sometimes seem rather misleading.</para>

<para>This manual page pertains to version &bz-version; of
<computeroutput>bzip2</computeroutput>.  Compressed data created
by this version is entirely forwards and backwards compatible
with the previous public releases, versions 0.1pl2, 0.9.0 and
0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following exception: 0.9.0
and above can correctly decompress multiple concatenated
compressed files.  0.1pl2 cannot do this; it will stop after
decompressing just the first file in the stream.</para>

<para><computeroutput>bzip2recover</computeroutput> versions
prior to 1.0.2 used 32-bit integers to represent bit positions in
compressed files, so it could not handle compressed files more
than 512 megabytes long.  Versions 1.0.2 and above use 64-bit ints
on some platforms which support them (GNU supported targets, and
Windows). To establish whether or not
<computeroutput>bzip2recover</computeroutput> was built with such
a limitation, run it without arguments. In any event you can
build yourself an unlimited version if you can recompile it with
<computeroutput>MaybeUInt64</computeroutput> set to be an
unsigned 64-bit integer.</para>

</sect1>



<sect1 id="author" xreflabel="AUTHOR">
<title>AUTHOR</title>

<para>Julian Seward,
<computeroutput>&bz-email;</computeroutput></para>

<para>The ideas embodied in
<computeroutput>bzip2</computeroutput> are due to (at least) the
following people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for the
Huffman coder), Peter Fenwick (for the structured coding model in
the original <computeroutput>bzip</computeroutput>, and many
refinements), and Alistair Moffat, Radford Neal and Ian Witten
(for the arithmetic coder in the original
<computeroutput>bzip</computeroutput>).  I am much indebted for