llbackup.8

lustre 1.6.5 source code

.\" -*- nroff -*-
.\" Copyright 2008 by Sun Microsystems.  All Rights Reserved.
.\" This file may be copied under the terms of the GNU Public License, v2.
.\"
.TH llbackup 8 "2008 Mar 15" Lustre "System management commands"
.SH NAME
llbackup \- backup a list of files, optionally from multiple nodes at once
.SH SYNOPSIS
.B llbackup -c
.RB [ -hjvxz "] [" -C
.IR directory ]
.RB [ -e
.IR rsh ]
.RB [ -i
.IR inputlist ]
.RB [ -l
.IR logdir ]
.RB [ -n
.IR nodes ]
.RB [ -s
.IR splitmb ]
.RB [ -S
.IR splitcount ]
.RB [ -T
.IR tar ]
.BI -f outputfilebase
.br

.B llbackup
.RB { -x | -t }
.RB [ -chjvz "] [" -C
.IR directory ]
.RB [ -e
.IR rsh ]
.RB [ -l
.IR logdir ]
.RB [ -n
.IR nodes ]
.RB [ -T
.IR tar ]
.BI -f tarfilelist
.SH DESCRIPTION
.B llbackup
takes a list of files to backup or restore, either from standard input, or
from a list of files supplied as a parameter and produces one or more archive
files containing the files.
.B llbackup
uses the
.BR tar (1)
or
.BR htar (1)
programs to produce archive files, and has many of the same options (though
not identical in some cases).  The reason for using llbackup instead of those
programs directly is that llbackup can run in parallel on multiple
nodes to do high-speed backups on clustered or distributed filesystems and
will split the work among multiple client nodes to increase performance.
.br

The archive files produced by
.B llbackup
are identical to normal tar files (having been created by the local tar
command) and can be extracted or listed by
.B tar
directly.
.br

The clients will pause and print status messages if they think there is not
enough free space in the output directory to avoid running out of space.
Backup tarfiles should be moved to offline storage or otherwise purged to
allow the backups to continue.  If the client is doing a restore it will
similarly pause and print status messages if the tarfile it was asked to
restore is not available on that node.
.SH OPERATIONS
The following options control what operation
.B llbackup
will perform:
.TP
.B -c
create archive.  This is the default operation if no other is given.
.TP
.B -x
extract tarfiles from the
.I tarfilelist
given to
.B -f
instead of backing them up.
.TP
.B -t
list table of contents of the
.I tarfilelist
given by
.BR -f .
.SH OPTIONS
.TP
.BI -C directory
change the backup command to run in
.I directory
instead of the current working directory so filenames given in the input
file are backed up or restored relative to
.IR directory .
Note that this affects all pathname components used during backup and restore
so the
.I inputlist
and
.I outputbase
should be specified by absolute pathname if they are not also in
.IR directory .
.TP
.BI -e rsh
specify the passwordless remote shell command (default
.BR ssh (1)).
.TP
.BI -f outputbase
specify base output filename for backup tar archives, to which a suffix will
be appended to ensure that multiple clients doing backup into the same
directory do not have conflicting filenames.  The
.I outputbase
file can be a tape or disk device, but must be the same device on each client
running the backup, in which case only a single process can run on each client.
Specifying
.B -
means write to standard output.
.TP
.BI -f tarfilelist
specify a file containing a list of tarfiles, one per line, to restore or
list the table of contents of.  Specifying
.B -
means read the list of tarfiles from standard input.  The
.I tarfilelist
can also be a tape or disk device, but since the individual tarfiles are
sent to individual clients, the same files must be accessible on all clients.
.TP
.B -h
print this help message and exit.  Use
.B -x
.B -h
or
.B -t
.B -h
to get the help options for restore or list operations.
.TP
.BI -i inputlist
list of files to backup when creating an archive.  If no
.B -i
option is specified or if
.I inputlist
is - then the list of files to back up is read from standard input.  The input
file list is a list of pathnames, one file per line, as generated by
.BR find (8),
.BR "lfs find" (8),
or
.BR e2scan (8)
for example.  For restore this option is currently unused.
.TP
.B -j
use
.BR bzip2 (1)
compression on input or output tar file(s).  This option is not
currently supported by the underlying
.BR htar (8)
implementation.
.TP
.BI -l logdir
directory for output logs.
.TP
.BI -n nodes
comma-separated list of client nodes to run backups.  It is possible to
specify the same client node multiple times in order to have multiple
instances of the backup or restore running in parallel on that node.
This is useful to maximize the bandwidth of the backup or restore process
and avoid client nodes underutilizing their bandwidth while generating
file lists or compressing files.
.TP
.BI -s splitmb
target size for backup chunks in megabytes (default 8192MiB).  Increasing this
size can improve performance and compression ratios, but may cause
underutilization of clients if the files are not evenly distributed.
This option has no meaning for restore or list operations.
.TP
.BI -S splitcount
number of files sent to each client before sending files to the next client
(default 200).  Increasing the
.I splitcount
keeps more related files together in individual tar files (meaning fewer
archives need to be processed when restoring files for a particular
subdirectory), but can lead to performance imbalance if files vary greatly
in size and one client has to back up many of them due to proximity of
filenames in the input list, while other clients back up smaller files.
Decreasing the
.I splitcount
means a more uniform distribution of files and usage among clients, but
means nearly every archive will have to be extracted to get only a subset
of files needed.
This option has no meaning for restore or list operations.
.TP
.BI -T tar
specify the backup command (default tar).  Optionally the
.BI htar (8)
command can be used to back up directly to an HPSS archive, if available.
.TP
.B -v
be verbose - list all files being processed.  By default, individual files
being backed up or restored are only reported in the log files because they
would all be intermixed coming from multiple clients in parallel and the
overall progress would be hard to determine.
.TP
.B -V
print version number and exit.
.TP
.B -z
use gzip compression on input or output tar file(s).  This option is not
currently supported by the underlying
.BR htar (8)
implementation.
.SH EXAMPLES
.TP
.B lfs find /home -mtime -7 | llbackup -c -n c1,c2 -f /bkup/bk2008-03-15
Create a backup of the
.B /home
filesystem finding files modified in the last 7 days, compress the output
using
.B gzip
and run on client nodes
.B c1
nodes, putting the output tar files into the
.B /bkup
directory using filenames starting with
.BR bk2008-03-15 .
.TP
.B find /bkup -name bk2007-11-28* | llbackup -x -n c1,c1,c2,c2 -C /tmp -f-
Restore
.B gzip
compressed files from the
.B bk2007-11-28
backup from the
.B /bkup
directory, using two restore processes on each of the client nodes
.B c1
and
.BR c2 ,
reading the list of tar files to restore from standard input and restoring
into the
.B /tmp
directory.
.SH BUGS
It is currently not possible to extract only a subset of files from the
tarfiles.  If that is necessary, then tar should be run directly on the
individual tar files after checking in the backup logs which tarfiles
have the relevant files.
.SH SEE ALSO
.BR lustre (7),
.BR tar (8),
.BR htar (8),
.BR ssh (1),
.BR rsh (1),
.BR find (1),
.BR lfs "(1) " find ,
.BR e2scan (8).