faq.html

linux 下的线程库源码

<CODE>read()</CODE> returns -1 and looks up errno.  Since
<CODE>_REENTRANT</CODE> is not defined, the reference to errno
accesses the global errno variable, which is most likely 0.  Hence the
code concludes that it cannot handle the error and stops.<P>

<H4><A NAME="H.4">H.4: With LinuxThreads, I can no longer use the signals
<code>SIGUSR1</code> and <code>SIGUSR2</code> in my programs! Why? </A></H4>

LinuxThreads needs two signals for its internal operation.
One is used to suspend and restart threads blocked on mutex, condition
or semaphore operations.  The other is used for thread cancellation.
Since the only two signals not reserved for the Linux kernel are
<code>SIGUSR1</code> and <code>SIGUSR2</code>, LinuxThreads has no
other choice than using them.  I know this is unfortunate, and hope
this problem will be addressed in future Linux kernels, either by
freeing some of the regular signals (unlikely), or by providing more
than 32 signals (as per the POSIX 1003.1b realtime extensions).<P>

In the meantime, you can try to use kernel-reserved signals either in
your program or in LinuxThreads.  For instance,
<code>SIGSTKFLT</code> and <code>SIGUNUSED</code> appear to be
unused in the current Linux kernels for the Intel x86 architecture.
To use these in LinuxThreads, the only file you need to change
is <code>internals.h</code>, more specifically the two lines:
<PRE>
        #define PTHREAD_SIG_RESTART SIGUSR1
        #define PTHREAD_SIG_CANCEL SIGUSR2
</PRE>
Replace them by e.g.
<PRE>
        #define PTHREAD_SIG_RESTART SIGSTKFLT
        #define PTHREAD_SIG_CANCEL SIGUNUSED
</PRE>
Warning: you're doing this at your own risks.<P>

<H4><A NAME="H.5">H.5: Is the stack of one thread visible from the
other threads?  Can I pass a pointer into my stack to other threads?
</A></H4>

Yes, you can -- if you're very careful.  The stacks are indeed visible
from all threads in the system.  Some non-POSIX thread libraries seem
to map the stacks for all threads at the same virtual addresses and
change the memory mapping when they switch from one thread to
another.  But this is not the case for LinuxThreads, as it would make
context switching between threads more expensive, and at any rate
might not conform to the POSIX standard.<P>

So, you can take the address of an "auto" variable and pass it to
other threads via shared data structures.  However, you need to make
absolutely sure that the function doing this will not return as long
as other threads need to access this address.  It's the usual mistake
of returning the address of an "auto" variable, only made much worse
because of concurrency.  It's much, much safer to systematically
heap-allocate all shared data structures. <P>

<HR>
<P>

<H2><A NAME="I">I.  X-Windows and other libraries</A></H2>

<H4><A NAME="I.1">I.1: My program uses both Xlib and LinuxThreads.
It stops very early with an "Xlib: unknown 0 error" message.  What
does this mean? </A></H4>

That's a prime example of the errno problem described in question <A
HREF="#H.2">H.2</A>.  The binaries for Xlib you're using have not been
compiled with <CODE>-D_REENTRANT</CODE>.  It happens Xlib contains a
piece of code very much like the one in question <A
HREF="#H.2">H.2</A>.  So, your Xlib fetches the error code from the
wrong errno location and concludes that an error it cannot handle
occurred.<P>

<H4><A NAME="I.2">I.2: So, what can I do to build a multithreaded X
Windows client? </A></H4>

The best solution is to recompile the X libraries with multithreading
options set.  They contain optional support for multithreading; it's
just that all binary distributions for Linux were built without this
support.  See the file <code>README.Xfree3.3</code> in the LinuxThreads
distribution for patches and info on how to compile thread-safe X
libraries from the Xfree3.3 distribution.  The Xfree3.3 sources are
readily available in most Linux distributions, e.g. as a source RPM
for RedHat.  Be warned, however, that X Windows is a huge system, and
recompiling even just the libraries takes a lot of time and disk
space.<P>

Another, less involving solution is to call X functions only from the
main thread of your program.  Even if all threads have their own errno
location, the main thread uses the global errno variable for its errno
location.  Thus, code not compiled with <code>-D_REENTRANT</code>
still "sees" the right error values if it executes in the main thread
only. <P>

<H4><A NAME="I.2">This is a lot of work. Don't you have precompiled
thread-safe X libraries that you could distribute?</A></H4>

No, I don't.  Sorry.  But you could approach the maintainers of
your Linux distribution to see if they would be willing to provide
thread-safe X libraries.<P>

<H4><A NAME="I.3">I.3: Can I use library FOO in a multithreaded
program?</A></H4>

Most libraries cannot be used "as is" in a multithreaded program.
For one thing, they are not necessarily thread-safe: calling
simultaneously two functions of the library from two threads might not
work, due to internal use of global variables and the like.  Second,
the libraries must have been compiled with <CODE>-D_REENTRANT</CODE> to avoid
the errno problems explained in question <A HREF="#H.2">H.2</A>.
<P>

<H4><A NAME="I.4">I.4: What if I make sure that only one thread calls
functions in these libraries?</A></H4>

This avoids problems with the library not being thread-safe.  But
you're still vulnerable to errno problems.  At the very least, a
recompile of the library with <CODE>-D_REENTRANT</CODE> is needed.
<P>

<H4><A NAME="I.5">I.5: What if I make sure that only the main thread
calls functions in these libraries?</A></H4>

That might actually work.  As explained in question <A HREF="#I.1">I.1</A>,
the main thread uses the global errno variable, and can therefore
execute code not compiled with <CODE>-D_REENTRANT</CODE>.<P>

<H4><A NAME="I.6">I.6: SVGAlib doesn't work with LinuxThreads.  Why?
</A></H4>

Because both LinuxThreads and SVGAlib use the signals
<code>SIGUSR1</code> and <code>SIGUSR2</code>.  One of the two should
be recompiled to use different signals.  See question <A
HREF="#H.4">H.4</A>.
<P>


<HR>
<P>

<H2><A NAME="J">J.  Signals and threads</A></H2>

<H4><A NAME="J.1">J.1: When it comes to signals, what is shared
between threads and what isn't?</A></H4>

Signal handlers are shared between all threads: when a thread calls
<CODE>sigaction()</CODE>, it sets how the signal is handled not only
for itself, but for all other threads in the program as well.<P>

On the other hand, signal masks are per-thread: each thread chooses
which signals it blocks independently of others.  At thread creation
time, the newly created thread inherits the signal mask of the thread
calling <CODE>pthread_create()</CODE>.  But afterwards, the new thread
can modify its signal mask independently of its creator thread.<P>

<H4><A NAME="J.2">J.2: When I send a <CODE>SIGKILL</CODE> to a
particular thread using <CODE>pthread_kill</CODE>, all my threads are
killed!</A></H4>

That's how it should be.  The POSIX standard mandates that all threads
should terminate when the process (i.e. the collection of all threads
running the program) receives a signal whose effect is to
terminate the process (such as <CODE>SIGKILL</CODE> or <CODE>SIGINT</CODE>
when no handler is installed on that signal).  This behavior makes a
lot of sense: when you type "ctrl-C" at the keyboard, or when a thread
crashes on a division by zero or a segmentation fault, you really want
all threads to stop immediately, not just the one that caused the
segmentation violation or that got the <CODE>SIGINT</CODE> signal.
(This assumes default behavior for those signals; see question
<A HREF="#J.3">J.3</A> if you install handlers for those signals.)<P>

If you're trying to terminate a thread without bringing the whole
process down, use <code>pthread_cancel()</code>.<P>

<H4><A NAME="J.3">J.3: I've installed a handler on a signal.  Which
thread executes the handler when the signal is received?</A></H4>

If the signal is generated by a thread during its execution (e.g. a
thread executes a division by zero and thus generates a
<CODE>SIGFPE</CODE> signal), then the handler is executed by that
thread.  This also applies to signals generated by
<CODE>raise()</CODE>.<P>

If the signal is sent to a particular thread using
<CODE>pthread_kill()</CODE>, then that thread executes the handler.<P>

If the signal is sent via <CODE>kill()</CODE> or the tty interface
(e.g. by pressing ctrl-C), then the POSIX specs say that the handler
is executed by any thread in the process that does not currently block
the signal.  In other terms, POSIX considers that the signal is sent
to the process (the collection of all threads) as a whole, and any
thread that is not blocking this signal can then handle it.<P>

The latter case is where LinuxThreads departs from the POSIX specs.
In LinuxThreads, there is no real notion of ``the process as a whole'':
in the kernel, each thread is really a distinct process with a
distinct PID, and signals sent to the PID of a thread can only be
handled by that thread.  As long as no thread is blocking the signal,
the behavior conforms to the standard: one (unspecified) thread of the
program handles the signal.  But if the thread to which PID the signal
is sent blocks the signal, and some other thread does not block the
signal, then LinuxThreads will simply queue in 
that thread and execute the handler only when that thread unblocks
the signal, instead of executing the handler immediately in the other
thread that does not block the signal.<P>

This is to be viewed as a LinuxThreads bug, but I currently don't see
any way to implement the POSIX behavior without kernel support.<P>

<H4><A NAME="J.3">J.3: How shall I go about mixing signals and threads
in my program? </A></H4>

The less you mix them, the better.  Notice that all
<CODE>pthread_*</CODE> functions are not async-signal safe, meaning
that you should not call them from signal handlers.  This
recommendation is not to be taken lightly: your program can deadlock
if you call a <CODE>pthread_*</CODE> function from a signal handler!
<P>

The only sensible things you can do from a signal handler is set a
global flag, or call <CODE>sem_post</CODE> on a semaphore, to record
the delivery of the signal.  The remainder of the program can then
either poll the global flag, or use <CODE>sem_wait()</CODE> and
<CODE>sem_trywait()</CODE> on the semaphore.<P>

Another option is to do nothing in the signal handler, and dedicate
one thread (preferably the initial thread) to wait synchronously for
signals, using <CODE>sigwait()</CODE>, and send messages to the other
threads accordingly.

<H4><A NAME="J.4">J.4: When one thread is blocked in
<CODE>sigwait()</CODE>, other threads no longer receive the signals
<CODE>sigwait()</CODE> is waiting for!  What happens? </A></H4>

It's an unfortunate consequence of how LinuxThreads implements
<CODE>sigwait()</CODE>.  Basically, it installs signal handlers on all
signals waited for, in order to record which signal was received.
Since signal handlers are shared with the other threads, this
temporarily deactivates any signal handlers you might have previously
installed on these signals.<P>

Though surprising, this behavior actually seems to conform to the
POSIX standard.  According to POSIX, <CODE>sigwait()</CODE> is
guaranteed to work as expected only if all other threads in the
program block the signals waited for (otherwise, the signals could be
delivered to other threads than the one doing <CODE>sigwait()</CODE>,
which would make <CODE>sigwait()</CODE> useless).  In this particular
case, the problem described in this question does not appear.<P>

One day, <CODE>sigwait()</CODE> will be implemented in the kernel,
along with others POSIX 1003.1b extensions, and <CODE>sigwait()</CODE>
will have a more natural behavior (as well as better performances).<P>

<HR>
<P>

<H2><A NAME="K">K.  Internals of LinuxThreads</A></H2>

<H4><A NAME="K.1">K.1: What is the implementation model for
LinuxThreads?</A></H4>

LinuxThreads follows the so-called "one-to-one" model: each thread is
actually a separate process in the kernel.  The kernel scheduler takes
care of scheduling the threads, just like it schedules regular
processes.  The threads are created with the Linux
<code>clone()</code> system call, which is a generalization of
<code>fork()</code> allowing the new process to share the memory
space, file descriptors, and signal handlers of the parent.<P>

Advantages of the "one-to-one" model include:
<UL>
<LI> minimal overhead on CPU-intensive multiprocessing (with
about one thread per processor);
<LI> minimal overhead on I/O operations;
<LI> a simple and robust implementation (the kernel scheduler does
most of the hard work for us).
</UL>
The main disadvantage is more expensive context switches on mutex and
condition operations, which must go through the kernel.  This is
mitigated by the fact that context switches in the Linux kernel are
pretty efficient.<P>

<H4><A NAME="K.2">K.2: Have you considered other implementation
models?</A></H4>

There are basically two other models.  The "many-to-one" model
relies on a user-level scheduler that context-switches between the
threads entirely in user code; viewed from the kernel, there is only
one process running.  This model is completely out of the question for
me, since it does not take advantage of multiprocessors, and require
unholy magic to handle blocking I/O operations properly.  There are
several user-level thread libraries available for Linux, but I found
all of them deficient in functionality, performance, and/or robustness.
<P>

The "many-to-many" model combines both kernel-level and user-level
scheduling: several kernel-level threads run concurrently, each
executing a user-level scheduler that selects between user threads.
Most commercial Unix systems (Solaris, Digital Unix, IRIX) implement
POSIX threads this way.  This model combines the advantages of both
the "many-to-one" and the "one-to-one" model, and is attractive
because it avoids the worst-case behaviors of both models --
especially on kernels where context switches are expensive, such as
Digital Unix.  Unfortunately, it is pretty complex to implement, and
requires kernel support which Linux does not provide.  Linus Torvalds
and other Linux kernel developers have always been pushing the
"one-to-one" model in the name of overall simplicity, and are doing a
pretty good job of making kernel-level context switches between
threads efficient.  LinuxThreads is just following the general
direction they set.<P>

<H4><A NAME="K.3">K.3: I looked at the LinuxThreads sources, and I saw
quite a lot of spinlocks and busy-waiting loops to acquire these
spinlocks.  Isn't this a big waste of CPU time?</A></H4>

Look more carefully.  Spinlocks are used internally to protect
LinuxThreads's data structures, but these locks are held for very
short periods of time: 10 instructions or so.  The probability that a
thread has to loop busy-waiting on a taken spinlock for more than,
say, 100 cycles is very, very low.  When a thread needs to wait on a
mutex, condition, or semaphore, it actually puts itself on a waiting
queue, then suspends on a signal, consuming no CPU time at all.  The
thread will later be restarted by sending it a signal when the state
of the mutex, condition, or semaphore changes.<P>

<HR>
<ADDRESS>[e-mail]</ADDRESS>
</BODY>
</HTML>