readme.db

一个C语言写的快速贝叶斯垃圾邮件过滤工具

Non-transactional databases do not store data that could help with
recovering data, so the recovery options are:

3.3.1 Salvage raw data

Note that your database may already have lost data, so this procedure
may not recover everything that used to be in the database before the
corruption happened.

To try this, suspend mail delivery to your directory, run these
commands:

  db_dump -r ~/.bogofilter/wordlist.db > ~/.bogofilter/wordlist.saved
  rm ~/.bogofilter/{__db.*,log.*,wordlist.db}
  db_load ~/.bogofilter/wordlist.db < ~/.bogofilter/wordlist.saved

and resume mail delivery.

3.3.2 Restore from backup

If the previous procedure has provided you with incomplete data, simply
replace your wordlist.db file by the latest good copy that you have.

3.3.3 Retrain from scratch

If you don't have good backups and the procedure from 3.3.1 also failed,
you have no other choice than to remove the wordlist.db file and redo
the whole training process with the usual bogofilter -s and bogofilter
-n commands.

3.4 COPYING DATABASES

If you intend to copy a transactional database, you MUST copy the *.db
file(s) first and then the log.* files. Note that the database should
NOT be subject to training while the copy is in progress, or automatic
log file removal should be suspended while the copy is made.

If you intend to copy a non-transactional database, you MUST NOT perform
any training while the copy is in progress.

Note that bogofilter's -u option can cause automatic training as mail
arrives.

If you need to copy the *.db files, DO NOT USE cp, BUT DO USE dd instead
and give it a block size that matches the data base's block size, which
can be found by running bogoutil --db-print-pagesize as shown in section
4.1 below. Example:

dd if=~/.bogofilter/wordlist.db bs=4096 of=/tmp/copy.db

The reason is that some operating systems let their cp command use
mmap() for efficiency without guaranteeing consistency of mmap() versus
simultaneous write().

A bf_copy script is provided for your convenience.

4. Other Information of Interest ---------------------------------------

4.1 GENERAL INFORMATION

Berkeley DB keeps some additional statistics about the database itself,
and in transactional databases also about caching, logging,
transactions and their efficiency.

The page size of the database file can be queried by bogoutil:

bogoutil --db-print-pagesize ~/.bogofilter/wordlist.db

Further statistics can be obtained by running the db_stat utility:

db_stat -h ~/.bogofilter -d wordlist.db # data base statistics

The following are only available in transactional mode:

db_stat -h ~/.bogofilter -e # environment statistics
db_stat -h ~/.bogofilter -m # buffer pool statistics
db_stat -h ~/.bogofilter -l # log statistics
db_stat -h ~/.bogofilter -t # transaction statistics

Note that statistics may disappear when the data base is recovered. They
will reappear after running bogofilter and are the more reliable the
more often bogofilter has been used since the last recovery.

You MUST NOT manually remove files named __db.NNN and log.NNNNNNNNNN
from the ~/.bogofilter directory.
================================================
REMOVING THESE FILES CAUSES DATABASE CORRUPTION!
================================================
These can contain update data for the data base that must be still
written back to the wordlist.db file - this happens when there are many
concurrent processes alongside a registration process.

You can safely remove the __db.NNN files with the following command, but
note that these will reappear when bogofilter or bogoutil is used:

  bogoutil --db-remove-environment ~/.bogofilter

log.* files are removed automatically in the default configuration, the
remaining ones must remain in place. For further information and on safe
removal, see section 3.1.

4.2 INTERESTING CONFIGURATION PARAMETERS FOR DB_CONFIG

A "DB_CONFIG" file that is in your bogofilter home directory, usually
~/.bogofilter, can be used to configure the data base behavior of
transactional databases. Some options take effect immediately, some need
'bogoutil --db-recover' to become effective. All options are ignored if
the database in the same directory as DB_CONFIG is non-transactional.

Here is a list of interesting settings, put one at a line, omitting the
leading spaces and hyphen:

SIZING OPTIONS:

 - set_cachesize      G B C
   (valid in Berkeley DB 3.1 - 4.4, requires recovery to change)

   sets the cache size to G gigabytes plus B bytes which are spread out
   in C equally sized caches (all figures are natural numbers). You
   cannot configure caches smaller than 20 kB, Berkeley DB will increase
   all caches under 500 MB by 25%, and you must provide at least one
   cache per 4 GB.

   This option takes precedence over Bogofilter's db_cachesize option!

   Example: set_cachesize 0 60000000 6
   will create six caches sized 12500000 bytes (12.5 MB, +25% applied)

 - set_lg_max         250000
   (valid in Berkeley DB 3.1 - 4.4, takes effect immediately)

   this option configures the maximum log file size, in bytes, before
   Berkeley DB starts a new log file. The default is 1 MB.

SAFE OPTIONS:

 - set_flags DB_DIRECT_DB
   (valid in Berkeley DB 4.1 - 4.4, takes effect immediately)

   this option turns off system buffering of *database* files, to avoid
   double caching of data. NOT SUPPORTED ON ALL PLATFORMS!

 - set_flags DB_DIRECT_LOG
   (valid in Berkeley DB 4.1 - 4.4, takes effect immediately)

   this option turns off system buffering of *log* files, to avoid
   double caching of data. NOT SUPPORTED ON ALL PLATFORMS!

 - set_flags DB_DSYNC_LOG
   (valid in Berkeley DB 4.3 - 4.4, takes effect immediately)

   this option can increase performance on some systems (and decrease on
   other systems), by using the O_DSYNC POSIX flag rather than a
   separate function to flush the logs.

 - set_flags DB_NOMMAP
   (valid in Berkeley DB 3.2 - 4.4, takes effect immediately)

   this option can reduce memory consumption at run time, particularly
   with large databases, at some cost of performance

 - set_flags DB_REGION_INIT
   (valid in Berkeley DB 3.2 - 4.4, takes effect immediately)

   this option causes all shared memory regions to be "page faulted"
   into core memory at application start and written at data base
   creation. This can improve performance under load, and it allows
   bogofilter or bogoutil to allocate disk space in advance, to avoid
   some out-of-space conditions later on.

 - set_verbose DB_VERB_CHKPOINT
   (valid in Berkeley DB 3.1 - 4.2, takes effect immediately)
 - set_verbose DB_VERB_DEADLOCK
   (valid in Berkeley DB 3.1 - 4.4, takes effect immediately)
 - set_verbose DB_VERB_RECOVERY
   (valid in Berkeley DB 3.1 - 4.4, takes effect immediately)
 - set_verbose DB_VERB_WAITSFOR
   (valid in Berkeley DB 3.1 - 4.4, takes effect immediately)

   these verbose flags cause extended output for long-lasting
   operations, ...CHKPOINT prints location information when searching
   for checkpoints, ...DEADLOCK prints information on deadlock
   detection, ...RECOVERY prints information during recovery and
   ...WAITSFOR prints the waits-for table during deadlock detection.

UNSAFE OPTIONS - these may impair robustness/recoverability of the data base

 - set_flags DB_TXN_NOSYNC
   (valid in Berkeley DB 3.3 - 4.4, takes effect immediately)

   if set, the log is not written or synchronously flushed at commit
   time. In case of an application or system crash, the last few
   registrations can be lost.

 - set_flags DB_TXN_WRITE_NOSYNC
   (valid in Berkeley DB 4.1 - 4.4, takes effect immediately)

   if set, the log is written, but not synchronously flushed at commit
   time. In case of a system crash, the last few registrations can be
   lost.

DANGEROUS OPTIONS - these can improve performance, but should be avoided

 - set_flags DB_TXN_NOT_DURABLE
   (valid in Berkeley DB 4.2 - 4.4, takes effect at next application start,
    replaced by DB_LOG_INMEMORY for version 4.3 - 4.4)

   this option prevents writing into log files. In case of application
   or system crashes, the data base can become corrupt, and large
   registrations can exhaust the log buffer space and then fail.

 - set_flags DB_LOG_INMEMORY
   (valid in Berkeley DB 4.3 - 4.4, takes effect at next application start)

   this option prevents the writing of any log files to disk. In case of
   application or system crashes, the data base can become corrupt, and
   large registrations can exhaust the log buffer space and then fail.

   After a crash, verify the data base (see below) and if it is corrupt,
   restore from backup or rebuild from scratch.

4.3 VERIFYING DATABASES TO CHECK FOR CORRUPTION

To verify that the database is intact, type:

    bogoutil --db-verify ~/.bogofilter/wordlist.db

There should be no output. If there are errors, you must recover your
database, see section 3.2 for methods.

A. Switching the disk drive's write cache off and on -------------------

A.1 Introduction

You need to determine the name of the disk device and its type.
Type "mount", you'll usually get an output that contains lines like
these; find the "on /home" if you have it, if you don't check for "on
/usr" if you have it, or finally, resort to looking at the "on /" line.

From this line, look at the left hand column, usually starting with /dev.

If you have FreeBSD, skip to section A.3 now.

A.2 Switching the write cache off or on in Linux

In this line you've found (see previous section A.1), you'll usually find
something that starts with /dev/hda, /dev/hde or /dev/sda in the left
hand column of that line, you can ignore the trailing number. /dev/hd*
means ATA, /dev/sd* means SCSI.

If the drive name starts with /dev/hd, type the following line, but
replace hda by hde or what else you may have found:

/sbin/hdparm -W0 /dev/hda
                 (replace -W0 by -W1 to re-enable the write cache)

If your drive name starts with /dev/sd, use the graphical scsi-config
utility and add a blank the device name on the command line; for
example:

scsi-config /dev/sda

You can "try changes" (they will be forgotten the next time the computer
is switched off) or "save changes" (settings will be saved permanently);
you can use the same utility to restore the previous setting or load
manufacturer defaults. Skip to section 2.4.

What is this scsi-config?

The scsi-config command is a Tk script, delivered with the scsiinfo
package.  At the time of writing, scsiinfo can be found at
ftp://tsx-11.mit.edu/pub/linux/ALPHA/scsi/scsiinfo-1.7.tar.gz .

For users who don't run X on their mail servers, there is also a
command-line utility, scsiinfo, in the package.  Setting parameters
with scsiinfo is a bit hairy, but the following sequence worked for two
of us who tried it (back up your drive first):

# get current disk settings and turn off the write cache
# (substitute the appropriate device for /dev/sda in all these commands)
parms=`scsiinfo -cX /dev/sda | sed 's/^./0/'`

# write the parameters back to the hard drive's current settings
# this needs to be put in a boot script
scsiinfo -cXR /dev/sda $parms

# if you don't want to put this in a boot script, you can alternatively
# save the parameters to the hard drive's settings area:
scsiinfo -cXRS /dev/sda $parms

You did back up your drive before trying that, right? :)


A.3 Switching the write cache off in FreeBSD

Have you read section A.1 already? You should have.

In this line you've found (see section A.1), you'll usually have a line
that starts with /dev/ad0, /dev/wd0 (either means you have ATA) or
/dev/da0 (which means you have SCSI).

If you have ATA, add the line

  hw.ata.wc="0"

to /boot/loader.conf.local, shut down all applications and reboot. (To
revert the change, remove the line, shut down all applications and
reboot.)

If you have SCSI, you'll need to decide if you want the setting until the next
reboot, or permanent (the permanent setting can be changed back, don't worry).
In either case, omit the leading /dev and trailing s<NUMBER><LETTER> parts
(/dev/da0s1a -> da0; /dev/da4s3f -> da4). Replace da0 by your device name in
these examples, and leave out the part in parentheses:

  camcontrol modepage da0 -m8 -e -P0 (effective until computer is switched off)
  camcontrol modepage da0 -m8 -e -P3 (save parameters permanently)

camcontrol will open a temporary file with a WCE: line on top. Edit the
figure to read 0 (cache disabled) or 1 (cache enabled), then save the
file and exit the editor.