relay.txt

linux 内核源代码

relay interface (formerly relayfs)
==================================

The relay interface provides a means for kernel applications to
efficiently log and transfer large quantities of data from the kernel
to userspace via user-defined 'relay channels'.

A 'relay channel' is a kernel->user data relay mechanism implemented
as a set of per-cpu kernel buffers ('channel buffers'), each
represented as a regular file ('relay file') in user space.  Kernel
clients write into the channel buffers using efficient write
functions; these automatically log into the current cpu's channel
buffer.  User space applications mmap() or read() from the relay files
and retrieve the data as it becomes available.  The relay files
themselves are files created in a host filesystem, e.g. debugfs, and
are associated with the channel buffers using the API described below.

The format of the data logged into the channel buffers is completely
up to the kernel client; the relay interface does however provide
hooks which allow kernel clients to impose some structure on the
buffer data.  The relay interface doesn't implement any form of data
filtering - this also is left to the kernel client.  The purpose is to
keep things as simple as possible.

This document provides an overview of the relay interface API.  The
details of the function parameters are documented along with the
functions in the relay interface code - please see that for details.

Semantics
=========

Each relay channel has one buffer per CPU, each buffer has one or more
sub-buffers.  Messages are written to the first sub-buffer until it is
too full to contain a new message, in which case it it is written to
the next (if available).  Messages are never split across sub-buffers.
At this point, userspace can be notified so it empties the first
sub-buffer, while the kernel continues writing to the next.

When notified that a sub-buffer is full, the kernel knows how many
bytes of it are padding i.e. unused space occurring because a complete
message couldn't fit into a sub-buffer.  Userspace can use this
knowledge to copy only valid data.

After copying it, userspace can notify the kernel that a sub-buffer
has been consumed.

A relay channel can operate in a mode where it will overwrite data not
yet collected by userspace, and not wait for it to be consumed.

The relay channel itself does not provide for communication of such
data between userspace and kernel, allowing the kernel side to remain
simple and not impose a single interface on userspace.  It does
provide a set of examples and a separate helper though, described
below.

The read() interface both removes padding and internally consumes the
read sub-buffers; thus in cases where read(2) is being used to drain
the channel buffers, special-purpose communication between kernel and
user isn't necessary for basic operation.

One of the major goals of the relay interface is to provide a low
overhead mechanism for conveying kernel data to userspace.  While the
read() interface is easy to use, it's not as efficient as the mmap()
approach; the example code attempts to make the tradeoff between the
two approaches as small as possible.

klog and relay-apps example code
================================

The relay interface itself is ready to use, but to make things easier,
a couple simple utility functions and a set of examples are provided.

The relay-apps example tarball, available on the relay sourceforge
site, contains a set of self-contained examples, each consisting of a
pair of .c files containing boilerplate code for each of the user and
kernel sides of a relay application.  When combined these two sets of
boilerplate code provide glue to easily stream data to disk, without
having to bother with mundane housekeeping chores.

The 'klog debugging functions' patch (klog.patch in the relay-apps
tarball) provides a couple of high-level logging functions to the
kernel which allow writing formatted text or raw data to a channel,
regardless of whether a channel to write into exists or not, or even
whether the relay interface is compiled into the kernel or not.  These
functions allow you to put unconditional 'trace' statements anywhere
in the kernel or kernel modules; only when there is a 'klog handler'
registered will data actually be logged (see the klog and kleak
examples for details).

It is of course possible to use the relay interface from scratch,
i.e. without using any of the relay-apps example code or klog, but
you'll have to implement communication between userspace and kernel,
allowing both to convey the state of buffers (full, empty, amount of
padding).  The read() interface both removes padding and internally
consumes the read sub-buffers; thus in cases where read(2) is being
used to drain the channel buffers, special-purpose communication
between kernel and user isn't necessary for basic operation.  Things
such as buffer-full conditions would still need to be communicated via
some channel though.

klog and the relay-apps examples can be found in the relay-apps
tarball on http://relayfs.sourceforge.net

The relay interface user space API
==================================

The relay interface implements basic file operations for user space
access to relay channel buffer data.  Here are the file operations
that are available and some comments regarding their behavior:

open()	    enables user to open an _existing_ channel buffer.

mmap()      results in channel buffer being mapped into the caller's
	    memory space. Note that you can't do a partial mmap - you
	    must map the entire file, which is NRBUF * SUBBUFSIZE.

read()      read the contents of a channel buffer.  The bytes read are
	    'consumed' by the reader, i.e. they won't be available
	    again to subsequent reads.  If the channel is being used
	    in no-overwrite mode (the default), it can be read at any
	    time even if there's an active kernel writer.  If the
	    channel is being used in overwrite mode and there are
	    active channel writers, results may be unpredictable -
	    users should make sure that all logging to the channel has
	    ended before using read() with overwrite mode.  Sub-buffer
	    padding is automatically removed and will not be seen by
	    the reader.

sendfile()  transfer data from a channel buffer to an output file
	    descriptor. Sub-buffer padding is automatically removed
	    and will not be seen by the reader.

poll()      POLLIN/POLLRDNORM/POLLERR supported.  User applications are
	    notified when sub-buffer boundaries are crossed.

close()     decrements the channel buffer's refcount.  When the refcount
	    reaches 0, i.e. when no process or kernel client has the
	    buffer open, the channel buffer is freed.

In order for a user application to make use of relay files, the
host filesystem must be mounted.  For example,

	mount -t debugfs debugfs /debug

NOTE:   the host filesystem doesn't need to be mounted for kernel
	clients to create or use channels - it only needs to be
	mounted when user space applications need access to the buffer
	data.


The relay interface kernel API
==============================

Here's a summary of the API the relay interface provides to in-kernel clients:

TBD(curr. line MT:/API/)
  channel management functions:

    relay_open(base_filename, parent, subbuf_size, n_subbufs,
               callbacks, private_data)
    relay_close(chan)
    relay_flush(chan)
    relay_reset(chan)

  channel management typically called on instigation of userspace:

    relay_subbufs_consumed(chan, cpu, subbufs_consumed)

  write functions:

    relay_write(chan, data, length)
    __relay_write(chan, data, length)
    relay_reserve(chan, length)

  callbacks:

    subbuf_start(buf, subbuf, prev_subbuf, prev_padding)
    buf_mapped(buf, filp)
    buf_unmapped(buf, filp)
    create_buf_file(filename, parent, mode, buf, is_global)
    remove_buf_file(dentry)

  helper functions:

    relay_buf_full(buf)
    subbuf_start_reserve(buf, length)


Creating a channel
------------------

relay_open() is used to create a channel, along with its per-cpu
channel buffers.  Each channel buffer will have an associated file
created for it in the host filesystem, which can be and mmapped or
read from in user space.  The files are named basename0...basenameN-1
where N is the number of online cpus, and by default will be created
in the root of the filesystem (if the parent param is NULL).  If you
want a directory structure to contain your relay files, you should
create it using the host filesystem's directory creation function,
e.g. debugfs_create_dir(), and pass the parent directory to
relay_open().  Users are responsible for cleaning up any directory
structure they create, when the channel is closed - again the host
filesystem's directory removal functions should be used for that,
e.g. debugfs_remove().

In order for a channel to be created and the host filesystem's files
associated with its channel buffers, the user must provide definitions
for two callback functions, create_buf_file() and remove_buf_file().
create_buf_file() is called once for each per-cpu buffer from
relay_open() and allows the user to create the file which will be used
to represent the corresponding channel buffer.  The callback should
return the dentry of the file created to represent the channel buffer.
remove_buf_file() must also be defined; it's responsible for deleting
the file(s) created in create_buf_file() and is called during
relay_close().

Here are some typical definitions for these callbacks, in this case
using debugfs:

/*
 * create_buf_file() callback.  Creates relay file in debugfs.
 */
static struct dentry *create_buf_file_handler(const char *filename,
                                              struct dentry *parent,
                                              int mode,
                                              struct rchan_buf *buf,
                                              int *is_global)
{
        return debugfs_create_file(filename, mode, parent, buf,
	                           &relay_file_operations);
}

/*
 * remove_buf_file() callback.  Removes relay file from debugfs.
 */
static int remove_buf_file_handler(struct dentry *dentry)
{
        debugfs_remove(dentry);

        return 0;
}

/*