locking.txt

SIP 1.5.0源代码

A useful experiment would be to kill them and see if imapd is affected
in any way, but I decline to do so without an OK from UCS!  ;-) If
killing statd/lockd can be done without breaking fcntl() on local
files, this would become one of the primary means of dealing with this
problem.

     The statd and lockd daemons have quite a reputation for extreme
fragility.  There have been numerous reports about the locking
mechanism being wedged on a systemwide or even clusterwide basis,
requiring a reboot to clear.  It is rumored that this wedge, once it
happens, also blocks local locking.  Presumably killing and restarting
statd would suffice to clear the wedge, but I haven't verified this.

     There appears to be a limit to how many locks may be in use at a
time on the system, although the documentation only mentions it in
passing.  On some of their systems, UCS has increased lockd's ``size
of the socket buffer'', whatever that means.

C-CLIENT USAGE

     c-client uses flock().  On System V systems, flock() is simulated
by an emulator that calls fcntl().  This emulator is provided by some
systems (e.g. AIX), or uses c-client's flock.c module.


BEZERK AND MMDF

     Locking in the traditional UNIX formats was largely dictated by
the status quo in other applications; however, additional protection
is added against inadvertantly running multiple instances of a
c-client application on the same mail file.

     (1) c-client attempts to create a .lock file (mail file name with
``.lock'' appended) whenever it reads from, or writes to, the mail
file.  This is an exclusive lock, and is held only for short periods
of time while c-client is actually doing the I/O.  There is a 5-minute
timeout for this lock, after which it is broken on the presumption
that it is a stale lock.  If it can not create the .lock file due to
an EACCES (protection failure) error, it once silently proceeded
without this lock; this was for systems which protect /usr/spool/mail
from unprivileged processes creating files.  Today, c-client reports
an error unless it is built otherwise.  The purpose of this lock is to
prevent against unfavorable interactions with mail delivery.

     (2) c-client applies a shared flock() to the mail file whenever
it reads from the mail file, and an exclusive flock() whenever it
writes to the mail file.  This lock is freed as soon as it finishes
reading.  The purpose of this lock is to prevent against unfavorable
interactions with mail delivery.

     (3) c-client applies an exclusive flock() to a file on /tmp
(whose name represents the device and inode number of the file) when
it opens the mail file.  This lock is maintained throughout the
session, although c-client has a feature (called ``kiss of death'')
which permits c-client to forcibly and irreversibly seize the lock
from a cooperating c-client application that surrenders the lock on
demand.  The purpose of this lock is to prevent against unfavorable
interactions with other instances of c-client (rewriting the mail
file).

     Mail delivery daemons use lock (1), (2), or both.  Lock (1) works
over NFS; lock (2) is the only one that works on sites that protect
/usr/spool/mail against unprivileged file creation.  Prudent mail
delivery daemons use both forms of locking, and of course so does
c-client.

     If only lock (2) is used, then multiple processes can read from
the mail file simultaneously, although in real life this doesn't
really change things.  The normal state of locks (1) and (2) is
unlocked except for very brief periods.


TENEX AND MTX

     The design of the locking mechanism of these formats was
motivated by a design to enable multiple simultaneous read/write
access.  It is almost the reverse of how locking works with
bezerk/mmdf.

     (1) c-client applies a shared flock() to the mail file when it
opens the mail file.  It upgrades this lock to exclusive whenever it
tries to expunge the mail file.  Because of the flock() bug that
upgrading a lock actually releases it, it will not do so until it has
acquired an exclusive lock (2) first.  The purpose of this lock is to
prevent against expunge taking place while some other c-client has the
mail file open (and thus knows where all the messages are).

     (2) c-client applies a shared flock() to a file on /tmp (whose
name represents the device and inode number of the file) when it
parses the mail file.  It applies an exclusive flock() to this file
when it appends new mail to the mail file, as well as before it
attempts to upgrade lock (1) to exclusive.  The purpose of this lock
is to prevent against data being appended while some other c-client is
parsing mail in the file (to prevent reading of incomplete messages).
It also protects against the lock-releasing timing race on lock (1).

OBSERVATIONS

     In a perfect world, locking works.  You are protected against
unfavorable interactions with the mailer and against your own mistake
by running more than one instance of your mail reader.  In tenex/mtx
formats, you have the additional benefit that multiple simultaneous
read/write access works, with the sole restriction being that you
can't expunge if there are any sharers of the mail file.

     If the mail file is NFS-mounted, then flock() locking is a silent
no-op.  This is the way BSD implements flock(), and c-client's
emulation of flock() through fcntl() tests for NFS files and
duplicates this functionality.  There is no locking protection for
tenex/mtx mail files at all, and only protection against the mailer
for bezerk/mmdf mail files.  This has been the accepted state of
affairs on UNIX for many sad years.

     If you can not create .lock files, it should not affect locking,
since the flock() locks suffice for all protection.  This is, however,
not true if the mailer does not check for flock() locking, or if the
the mail file is NFS-mounted.

     What this means is that there is *no* locking protection at all
in the case of a client using an NFS-mounted /usr/spool/mail that does
not permit file creation by unprivileged programs.  It is impossible,
under these circumstances, for an unprivileged program to do anything
about it.  Worse, if EACCES errors on .lock file creation are no-op'ed
, the user won't even know about it.  This is arguably a site
configuration error.

     The problem with not being able to create .lock files exists on
System V as well, but the failure modes for flock() -- which is
implemented via fcntl() -- are different.

     On System V, if the mail file is NFS-mounted and either the
client or the server lacks a functioning statd/lockd pair, then the
lock attempt would have hung forever if it weren't for the fact that
c-client tests for NFS and no-ops the flock() emulator in this case.
Systemwide or clusterwide failures of statd/lockd have been known to
occur which cause all locks in all processes to hang (including
local?).  Without the special NFS test made by c-client, there would
be no way to request BSD-style no-op behavior, nor is there any way to
determine that this is happening other than the system being hung.

     The additional locking introduced by c-client was shown to cause
much more stress on the System V locking mechanism than has
traditionally been placed upon it.  If it was stressed too far, all
hell broke loose.  Fortunately, this is now past history.

TRADEOFFS

     c-client based applications have a reasonable chance of winning
as long as you don't use NFS for remote access to mail files.  That's
what IMAP is for, after all.  It is, however, very important to
realize that you can *not* use the lock-upgrade feature by itself
because it releases the lock as an interim step -- you need to have
lock-upgrading guarded by another lock.

     If you have the misfortune of using System V, you are likely to
run into problems sooner or later having to do with statd/lockd.  You
basically end up with one of three unsatisfactory choices:
	1) Grit your teeth and live with it.
	2) Try to make it work:
	   a) avoid NFS access so as not to stress statd/lockd.
	   b) try to understand the code in statd/lockd and hack it
	      to be more robust.
	   c) hunt out the system limit of locks, if there is one,
	      and increase it.  Figure on at least two locks per
	      simultaneous imapd process and four locks per Pine
	      process.  Better yet, make the limit be 10 times the
	      maximum number of processes.
	   d) increase the socket buffer (-S switch to lockd) if
	      it is offered.  I don't know what this actually does,
	      but giving lockd more resources to do its work can't
	      hurt.  Maybe.
	3) Decide that it can't possibly work, and turn off the 
	   fcntl() calls in your program.
	4) If nuking statd/lockd can be done without breaking local
	   locking, then do so.  This would make SVR4 have the same
	   limitations as BSD locking, with a couple of additional
	   bugs.
	5) Check for NFS, and don't do the fcntl() in the NFS case.
	   This is what c-client does.

     Note that if you are going to use NFS to access files on a server
which does not have statd/lockd running, your only choice is (3), (4),
or (5).  Here again, IMAP can bail you out.

     These problems aren't unique to c-client applications; they have
also been reported with Elm, Mediamail, and other email tools.

     Of the other two SVR4 locking bugs:

     Programmer awareness is necessary to deal with the bug that you
can not get an exclusive lock unless the file is open for write.  I
believe that c-client has fixed all of these cases.

     The problem about opening a second designator smashing any
current locks on the file has not been addressed satisfactorily yet.
This is not an easy problem to deal with, especially in c-client which
really doesn't know what other files/streams may be open by Pine.

     Aren't you so happy that you bought an System V system?