howto

一套客户/服务器模式的备份系统代码,跨平台,支持linux,AIX, IRIX, FreeBSD, Digital Unix (OSF1), Solaris and HP-UX.

                AF's Backup HOWTO
                =================


Index
-----

1. How to optimize the performance to obtain a short backup time ?

2. How to start the backup on several hosts from a central machine ?

3. How to store the backup in a filesystem instead of a tape ?

4. How to use several streamer devices on one machine ?

5. How to recover from a server crash during backup ?

6. How to port to other operating systems ?

7. How to provide recovery from hard crashes (disk crash, ...) ?

8. How to make differential backups ?

9. How to use several servers for one client ?

10. How can i automatically make copies of the written tapes after a backup ?

11: How to redirect network backups through a secure ssh connection ?

12: What's the appropriate way to eject the cartridge after backup ?

13: How to encrypt the stored files and not only compress them ?

14: How to use the multi-stream server ? Anything special there ?

15: How many clients can connect the multi-stream server ?

16: How to get out of the trouble, when the migration script fails ?

17: How to use built-in compression ?

18: How to save database contents ?

19: How to use the ftape driver ?

20: How to move a cartridge to another set due to it's usage count ?

21: How to make backups to different cartridge sets by type or by date ?

22: How to achieve independence from the machine names ?

23: How to restrict the access to cartridges for certain clients ?

24: How to recover from disaster (everything is lost) ?

25: How to label a tape, while the server is waiting for a tape ?

26: How to use a media changer ?

27: How to build Debian packages ?

28: How to let users restore on a host, they may not login to ?

29: How to backup through a firewall ?

30: How to configure xinetd for afbackup ?

31: How to redirect access, when a client contacts the wrong server ?

32: How to perform troubleshooting when encountering problems ?

33: How to use an IDE tape drive with Linux the best way ?

34: How to make afbackup reuse/recycle tapes automatically ?

35: How to make the server speak one other of the supported languages ?

36: How to build a Solaris package of the afbackup software ?


D. Do-s and Dont-s

F. The FAQ


--------------------------------------------------------------------------

1. How to optimize the performance to obtain a short backup time ?

Basically since version 2.7 the client side tries to optimally adapt
to the currently maximum achievable throughput, so the administrator
doesn't have to do much here.
The crucial point is the location of the bottleneck for the throughput
of the backup data stream. This can be one of:

- The streamer device
- The network connection between backup client and server
- The CPU on the backup client (in case of compression selected)

What usually is not a problem:

- The CPU load of the server

The main influence the administrator has on a good backup performance
is the compression rate on the client side. In most cases the bottleneck
for the data stream will be the network. If it is based on standard
ethernet, the maximum throughput without any other network load will be
around 1 MB/sec. With 100 MBit ethernet or a similar technology about
10 MB/sec might be achieved, so the streamer device is probably the
slowest part (with maybe 5 MB/sec for a Exabyte tape). To use this
capacity it is not clever to plug up the client side CPU with heavy
data compression load. This might be inefficient and thus lead to a
lousy backup performance. The influence of the compression rate on the
backup performance can be made clear with the following table. The
times in seconds have been measured with the (unrepresentative)
configuration given below the table. The raw backup duration gives the
pure data transmission time without tape reeling or cartridge loading
or unloading.

 compression program   |  raw backup duration
-----------------------+----------------------
  gzip -1              |    293 seconds         |
  gzip -5              |    334 seconds         |
  compress             |    440 seconds         | increasing duration
  <no compression>     |    560 seconds         |
  gzip -9              |    790 seconds         V


Configuration:
Server/Client machine:
  586, 133/120MHz (server/client), 32/16 MB (server/client)
Network:
  Standard ethernet (10 MB, 10BASE2 (BNC/Coax), no further load)
Streamer:
  HP-<something>, 190 kByte/sec

Obviously the bottleneck in this configuration is the streamer.
Anyway it shows the big advantage compression can have on the
overall performance. The best performance is here achieved with
the lowest compression rate and thus the fastest compression
program execution. I would expect, that the performance optimum
shifts towards a somewhat better compression with a faster client
CPU (e.g. the latest Alpha-rocket).

So to find an individual performance optimum i suggest to run some
backups with a typical directory containing files and subdirectories
of various sizes. Run these backups manually on the client-side machine
with different compression ratios using the "client"-command as
follows:

/the/path/to/bin/afclient -cvnR -h your_backuphost -z "gzip -1" gunzip \
                             /your/example/directory

Replace "gzip -1" and "gunzip" appropriately for the several runs.


--------------------------------------------------------------------------

2. How to start the backup on several hosts from a central machine ?

For this purpose serves the remote startup utility. To implement this
as fast as possible, a part of the serverside installation must be
made on the client side, where it is requested to start the backup from
a remote site. Choose the appropriate option when running the Install-
script or follow the instructions in the INSTALL file.

To start a backup on another machine, use the -X option of the
client-program. A typical invocation is

/the/path/to/client/bin/afclient -h <hostname> -X incr_backup

This starts an incremental backup on the supplied host. Often
-k /path/to/cryptkey must be given as well, if on the remote
side an EncryptionKeyFile is configured, what is recommended. Each
program on the remote host residing in the directory configured
as Program-Directory in the configuration file of the serverside
installation part of the remote host (default: $BASEDIR/server/rexec)
can be started, but no other. The entries may be symlinks, but
they must have the same filename like the programs, they point to.

The machine, where this script is started may be any machine in
the network having the client side of the backup system installed.


--------------------------------------------------------------------------

3. How to store the backup in a filesystem instead of a tape ?

There are several ways how to accomplish that. Two options are
explained here. I personnally prefer option 2, but they are
basically equivalent.

* Option 1 (using symbolic links)

Assumed the directory, where you'd like to store the backup, is
/var/backup/server/vol.X with X being the number of the pseudo-
cartridge, change to the directory /var/backup/server and create
a symbolic link and a directory like this:

 ln -s vol.1 vol ; mkdir vol.1

Then create the file `data.0' and a symlink `data' to it with

 touch vol/data.0
 ln -s data.0 vol/data

The directories and symlinks /var/backup/server/vol* must be owned
or at least be writable for the user, under whose ID the backup server
is running. The same applies for the directory /var/backup/server.
If this is not root, issue an appropriate chown command, e.g.:

 chown backup /var/backup/server /var/backup/server/vol*

At least two pseudo-cartridges should be used. This is achieved by
limiting the number of bytes to be stored on each of them. So now
edit your serverside configuration file and make e.g. the following
entries (assuming /usr/backup/server/bin is the directory, where the
programs of the server side reside):

Backup-Device:          /var/backup/server/vol/data
Tape-Blocksize:         1024
Cartridge-Handler:      1
Number Of Cartridges:	1000
Max Bytes Per File:     10485760
Max Bytes Per Tape:     104857600
Cart-Insert-Gracetime:  0
SetFile-Command:        /bin/rm -f %d;touch %d.%m; ln -s %d.%m %d; exit 0
SkipFiles-Command:      /usr/backup/server/bin/__inc_link -s %d %n
Set-Cart-Command:       /bin/rm -f /var/backup/server/vol; mkdir -p /var/backup/server/vol.%n ; ln -s vol.%n /var/backup/server/vol ; touch %d.0 ; /bin/rm -f %d ; ln -s data.0 %d;exit 0
Change-Cart-Command:    exit 0
Erase-Tape-Command:     /bin/rm -f %d.[0-9]* %d ; touch %d.0 ; ln -s %d.0 %d ; exit 0

If the directory /var/backup/server/vol/data is on a removable media,
you can supply the number of media you would like to use and an
eject-command as follows:

Number Of Cartridges:   10
# or whatever

Change-Cart-Command:    your_eject_command

If a suitable eject-command does not exist, try to write one yourself.
See below for hints.

Furthermore you can add the appropriate umount command before the eject-
command like this:

Change-Cart-Command:    umount /var/backup/server/vol ; your_eject_command

To get this working the backup serverside must run as root. Install the
backup stuff supplying the root-user when prompted for the backup user.
Or edit /etc/inetd.conf and replace backup (or whatever user you configured)
(5th column) with root, sending a kill -1 to the inetd afterwards.
Actually you must mount the media manually after having it inserted into
the drive. Afterwards run the command /path/to/server/bin/cartready to
indicate, that the drive is ready to proceed. This is the same procedure
like having a tape drive.

Each media you will use must be prepared creating the file "data.0" and
setting the symbolic link "data" pointing to data.0 like described above.


* Option 2 (supply a directory name as device)

Like with option 1 several pseudo-cartridges should be used, at
least two. Like above create a directory to contain the backup data
and a symlink, then chown them to the backup user:

 mkdir -p /var/backup/server/vol.1
 ln -s vol.1 /var/backup/server/vol
 chown backup /var/backup/server/vol*

Using one of the serverside configuration programs or editing the
configuration file, supply a directory name as the backup device.
The directory must be writable for the user, under whose ID the
server process is started (whatever you configured during
installation, see /etc/inetd.conf). The backup system then writes
files with automatically generated names into this directory.
The rest of the configuration should e. g. be set as follows:

Backup-Device:          /var/backup/server/vol
Tape-Blocksize:         1024
Cartridge-Handler:      1
Number Of Cartridges:   100
Max Bytes Per File:     10485760
Max Bytes Per Tape:     104857600
Cart-Insert-Gracetime:  0
SetFile-Command:        exit 0
SkipFiles-Command:      exit 0
Set-Cart-Command:       /bin/rm -f %d ; mkdir -p %d.%n ; ln -s %d.%n %d ;  exit 0
Change-Cart-Command:    exit 0
Erase-Tape-Command:     /bin/rm -f %d/* ; exit 0

A SetFile-Command is mandatory, so this exit 0 is a dummy.
For the further options (using mount or eject commands) refer
to the explanations under * Option 1.


(
   How to write an eject command for my removable media device ?

If the information in the man-pages is not sufficient or you don't
know, where to search, try the following:
Do a grep ignoring case for the words "eject", "offline" and
"unload" over all system header-files like this: