readme

Metalog is a modern replacement for syslogd and klogd. The logged messages can be dispatched accord

                               .:. METALOG .:.



           ------------------------ BLURB ------------------------


Metalog is a modern replacement for syslogd and klogd. The logged messages
can be dispatched according to their facility, urgency, program name and/or
Perl-compatible regular expressions. Log files can be automatically rotated
when they exceed a certain size or age. External shell scripts (e.g., mail)
can be launched when specific patterns are found.

Metalog is easier to configure than syslogd and syslog-ng, accepts unlimited
number of rules and has (switchable) memory bufferization for maximal
performance.


        ------------------------ COMPILATION ------------------------
        
        
In order to compile Metalog, you should have the PCRE library on your
system. That library is used to process Perl-compatible regular expressions.

libpcre comes with many distributions, but you can also download it from :

                             http://www.pcre.org
                             
Once installed, don't forget to run "ldconfig".

Then, compiling and installing Metalab isn't rocket science :

./configure
make install-strip

As of version 0.4, Metalog is only guaranteed to compile on GNU/Linux,
OpenBSD, FreeBSD and NetBSD.


       ------------------------ CONFIGURATION ------------------------
       
       
A configuration file called /etc/metalog.conf should be installed. You can
find a sample file in this directory, but it's certainly not perfect for
your system and your needs. So read on.

A configuration file has the following syntax :

[default values]

<section title> :
<section values>
...

<section title 2> :
<section 2 values>
...

[...]

To make it clear, here's an example :

maxsize  = 100000
maxtime  = 86400
maxfiles = 5

Kernel messages :

  facility = "kern"
  logdir   = "/var/log/kernel"

Crond :

  program  = "crond"
  logdir   = "/var/log/crond"

A section defines several things :

 - A title (useless for the software, it's just to make your configuration
file look better) .

 - Filters : you can define facilities, program names, urgency levels and
regular expressions. An incoming message will pass through all filters. If
all conditions are matching, actions defined for the section are performed.

 - Actions : they are taken only when all previous conditions are met. Only
two actions are currently possible : write the message to a log file, and/or
launch an external script.

Here's a list of values that can be independently assigned for each section :

* minimum = <level> : only record a message if its urgency is inferior or
equal to <level> . Level '0' is the most critical one, while level '7' is for
debugging messages. 'minimum = 5' will strip all non-important messages. The
default minimum level is 7 (ie. keep all messages) .

Example : record only critical messages to /var/log/important :

Critical messages :

  facility = "*"
  minimum  = 1
  logdir   = "/var/log/important"

* facility = <facility> : only record a message if the application that
issued it uses syslog facility <facility> . Facility names are : "auth",
"authpriv", "cron", "daemon", "ftp", "kern", "lpr", "mail", "news",
"security", "syslog", "user", "uucp", "local0", "local1" ... "local7" .
All kernel messages are logged with facility "kern". A section can have
several "facility = ..." lines to match more than one facility.

Example : record all authentication messages to /var/log/auth :

Authentication messages :

  facility = "auth"
  facility = "authpriv"
  logdir   = "/var/log/auth"
  
A wildcard sign is a special facility, that matches everything. You can use
it to redirect all messages to a directory, or in combination with other
filters.

* program = <program> : only record messages signed by a specific daemon or
program. You can use this as a modern alternative to syslog facilities : use
"*" as a facility, and set a program name with that directive. Kernel
messages can be caught with 'program = "kernel"' .

Example : record Pure-FTPd messages in a directory, and in.ftpd messages in
another directory (although the facility is the same : FTP) :

In.FTPd messages :

  facility = "*"
  program  = "in.ftpd"
  logdir   = "/var/log/in.ftpd"

Pure-FTPd messages :

  facility = "*"
  program  = "pure-ftpd"
  logdir   = "/var/log/pure-ftpd"

* regex = <regex> : a message must match the regular expression to pass that
filter. Multiple regexes are allowed for a single section. All Perl
extensions are allowed, and matching is case insensitive.

Example : record all authentication failures to /var/log/pwdfail :

Password failures :

Password failures :

  regex    = "(password|login|authentication)\s+(fail|invalid)"
  regex    = "(failed|invalid)\s+(password|login|authentication)"
  regex    = "ILLEGAL ROOT LOGIN"
  logdir   = "/var/log/pwdfail"

* maxsize = <size in bytes> : automatically rotate log files when their size
have exceeded that size. Messages are never truncated and no message can be
lost during a rotation. Every section can have a different maxsize value.

Example : rotate logs files when they are more than 1,000,000 bytes long :

  maxsize = 1000000

* maxfiles = <number of files> : how many files to keep after rotation. This
parameter can be set independently for each section. It defaults to 5.

* maxtime = <age in seconds> : automatically rotate log files when they are
older than this number of seconds. Every section can have a different
maxtime value. maxsize and maxtime can be combined together, so that rotation
occurs when any of these conditions occurs.

Example : rotate log files when they are more than 1,000,000 bytes long, and
every day, and keep only 3 history files :

  maxsize  = 1000000
  maxtime  = 86400
  maxfiles = 3

* logdir = <directory> : record messages in the specified directory. If the
directory doesn't exist, it will be automatically created when the first
matching message will be logged (the parent directory has to exist, though) .

* command = <path/to/command> : run a program or a shell-script when all
conditions are met. This directive is not incompatible with logdir : a
message can be both logged and passed to an external command. When the
command is launched, the first argument is filled with the date the message
was received, the second argument is the program name, and the last one is
the text of the message itself. Environment variables aren't cleared.

Example : send a mail to root when authentication failures occur :

Mail password failures :

  regex    = "(password|login|authentication)\s+(fail|invalid)"
  regex    = "(failed|invalid)\s+(password|login|authentication)"
  regex    = "ILLEGAL ROOT LOGIN"
  command  = "/usr/local/sbin/pwdfail.sh"
  
"pwdfail.sh" can be a simple shell script like this one :

#! /bin/sh

echo "$3" | mail -s "Password failure (program : $2)" root

  Don't forget to properly quote arguments to avoid security problems.
  
  
         ------------------------ LOG FILES ------------------------
         

With the "logdir" directive, messages from a specific section are recorded
in a directory. In that directory, the following files are created :

- "current" : this file contains the latest recorded messages. It can be
incomplete (ie. data is being written to the file when you are reading it) .

- ".timestamp" : the creation date of "current". Needed for the "maxtime"
directive.

- "log-<year>-<month>-<day>-<hour>:<minutes>:<seconds>" : old logs,
chronologically sorted.

If you ever delete these files by hand for some obscure reasons, it's not a
bad idea to restart the daemon (and even better : to stop it before you mess
the directories) .

  
      ------------------------ RUNNING METALOG ------------------------
      

Kill "klogd" and "syslogd" first. Don't run Metalog while they are running.
Multiple programs listening for the same source of data is a silly idea.

Once these programs are killed (and you are *sure* they are killed), check
that /etc/metalog.conf is installed, and simply run :

/usr/local/sbin/metalog &

On GNU/Linux systems, two processes are spawned : one is the "MASTER", doing
much of the work. The other one (named "KERNEL") is needed for Linux and
replaces the "klogd" daemon. It is responsible for logging kernel messages.

You may notice that messages aren't immediatly recorded into log files (but
scripts are run immediatly, though) . This is intentional : to avoid disk
I/O, and unlike traditional syslog daemons, Metalog works in memory buffers,
then flushes the data to disk by blocks. It improves a lot overall
performances.

If you want to watch activity in real-time (like a good old "tail -f" on a
log file), you can temporary disable buffering. Just send an USR1 signal to
the "MASTER" process. You can re-enable buffering afterwards, by sending an
USR2 signal.

  
     ------------------------ RUN-TIME OPTIONS ------------------------
      
      
Metalog accepts some run-time options :

- '-B' or '--daemonize' : have the server start in background
(daemonization) .

- '-c <xxx>' or '--consolelevel=<xxx>' : set the console log level on
Linux. Valid values are from 0 to 7. The default is 7.

- '-h' or '--help' : show help and version number.

- '-s' or '--sychronous' : start in synchronous mode, with no bufferization.






                        -Frank DENIS "Jedi/Sector One" <[e-mail]> .