todo

distcc编译器的源代码.好像是readhat公司开发的.

TODO list for distcc

See also TODO comments in source files and doc/todo/



convert web site from latte to say php

    Latte is nice, but seems to be gradually becoming unmaintained;
    not in Ubuntu or Gentoo.


failing to resolve a host should be a soft failure



State files depend on host byte order and break when DISTCC_DIR is
shared between heterogenous machines.  Of course sharing this
directory is probably a bad idea anyhow.



error messages get badly interleaved

    Large writes are not always atomic.  

    When running parallel compiles that produce many warnings/errors,
    the errors can get mixed up, both between lines and within lines.

    Someone suggested writing out through stdio, but I don't see why
    that would particularly help.

    This needs to be done even when writing to a file.

    It might be more useful to voluntarily write output one line at a
    time so as to increase the chance that each line is written atomically.
    


hardcode "gcc" not "cc"?

    I don't think distcc works with anything else.  So why take the
    risk of calling anything else?


monitor/state/lock files be on NFS

    What happens if the processes are missing?

    Should we perhaps specially handle files created by a remote
    machine, e.g. but the hostname in?

    Handle ESTALE or short EOF.

    http://groups.google.com/groups?selm=netappCJyvKo.MrI%40netcom.com




if connection fails, reschedule remotely?

    See messages from Heiko

    Perhaps if compilation on one remote machine fails, try another, rather
    than falling back to localhost?

    However, we do need to make sure that if all remote possibilities
    are eliminated, then we still run locally.

    Perhaps we should more carefully distinguish e.g. "failed to connect",
    "server dropped connection", etc etc.  

    Backing off from downed machines makes this a little
    less unnecessary.


auto-check socklen_t mess

    Dmitri says:           

> By the way, about the accept() argument type (int, size_t, or socklen_t)
> issue I had already reported in a previous post, an autoconf macro is
> available after all. See:
> http://www.gnu.org/software/ac-archive/htmldoc/ac_prototype_accept.html
> http://www.gnu.org/software/ac-archive/htmldoc/ac_func_accept_argtypes.html
> It would be better to fix this specific issue, as I think it could break
> 64-bit builds where the type of the argument is actually important.


    Perhaps it should be int if not defined.  See accept(2).


some kind of memory leak in gnome monitor?


"compiler not found"

    Perhaps distinguish this as a separate error case.  We need to
    trap the error from exec on the server, pass that back across the
    network, and then handle it specially on the client.

    Back off from the machine where it failed and retry locally?

    This would be more useful when we explicitly set compiler versions.


scheduler should allow for clock/bus speed

    (Perhaps front-side bus speed is dominant, since compiling won't
    fit in cache?)

    Suppose we have one 2GHz and one 1GHz machine.  Jobs will take
    roughly twice as long to run on the second one; conversely we can
    run two jobs on the first one in the time it takes to run one on
    the second machine.



gkrellm monitor for distcc

    Ought to work with client-server mode
 
    Possibly easier than writing everything ourselves

    Show number of running jobs?


don't change the path

    Rather than getting into this mess of changing the $PATH, perhaps
    we should just check more carefully at the moment that we execute
    things?

    One problem with this might be interaction with ccache.  If we
    have doubly masqueraded distcc:ccache:gcc, then ccache probably
    needs to see itself as the first item on the path to be able to
    find the right gcc.

    Perhaps we should remove items from the path, rather than trimming
    the path?

    It would be good to unify the code in dcc_support_masquerade()
    with dcc_trim_path().

    Perhaps distccd should do path munging when it gets a request
    rather than at startup.  It's ugly that the daemon's idea of the
    correct path may be wrong if files are changed after the daemon is
    started.


DEPENDENCIES_OUTPUT

    The problem is when the preprocessor and compiler are both run
    with this variable set.  The compiler appends directions for
    compiling from the temporary .i to the temporary .o, which is
    unwanted.

    This is a problem for ccache (now fixed), but not a problem for
    distcc because we always run the preprocessor and always locally.
    The remote compiler doesn't see the variable.


multiple cleanup calls at end

    why is this happening?


monitor

    There's two things that could be monitored.  First is the daemon running
    on `this' computer and the client that is sending processes across the
    network.

    lisa writes:
    
    Some things I'd like to see for the daemon:
    1) Uptime
    2) Configuration (port, lzo compression? ssh enabled? etc)
    3) Number of jobs done (and a spread of the types of errors reported)
    4) Average throughput
    5) Current compiling tasks (pid, Src, filesize, filename, time recv'd)


    For the client (ie, distccmon-gnome replacement):
    On a per job basis
    For SEND and RECEIVE state:
    1) Current throughput
    2) Type of connection (ssh? port? lzo?)
    3) pid
    4) filename
    For (remote) COMPILE state:
    1) The actual pre-processed filename
    2) Type of connection (ssh? port? lzo?)
    3) pid
    For (local) LINKING state:
    1) The actual pre-processed filename
    2) pid
    3) Where the object code was compiled
    4) The post-linked filenamed (gcc ... -o [display this])
    For (local) PREPROCESS state:
    1) filename
    2) pid

    A tall order to be sure, and it'd suck to do the GUI... but you asked.
    :)



unlink .i file as soon as it has been opened for sending

    Might help with vm performance by hinting to the kernel that it
    will be discarded.

    Possibly reduces the chances of temporary files being left behind.

    However, will not work on Windows.



variable to add extra remote cflags, to handle icpc



handle -xc++, etc

    Can be either one or two arguments.


handle -Wp,-MF

    Some makefiles seem to generate this.  Aarg!



Installation as an SSH subsystem

    Might make use easier on Windows.  I don't see any real advantage
    on Unix.
 
    I think the daemon should already be OK for this.  It just
    requires a slight change in the way we call ssh from the client.

    In fact, if you just wrote a small script that rearranged the
    arguments and put that in DISTCC_SSH then everything would
    probably be fine already.


control through command line

    Handle options like

       --distcc-verbose
       --distcc-hosts=

    to allow options to be set on the command line.

    I'm not sure this is a good idea or useful.


If we have produced a .i file and need to fall back to running locally
then use that rather than the original source.  On the other hand,
falling back to running the original command is possibly more robust.

 * @todo Make absolutely sure that if we fail, the .o file is removed.
 * Perhaps it would be better to receive to a temporary file and then
 * rename into place?  On the other hand, gcc seems to just write
 * directly, and if we fail or crash then Make ought to know not to
 * use it.
 *
 * @todo Count the preprocessor, and any compilations run locally, against the
 * load of localhost.  In doing this, make sure that we cannot deadlock
 * against a load limit, by having a case where we need to hold one lock and
 * take another to make progress.  I don't think there should be any such case
 * -- we can release the cpp lock before starting the main compiler.
 *

allow more control over verbosity

    For example, for the client, it would be nice to get just 'info'
    level messages about things that can or can't be distributed.


split gcc-specific argument parsing into a separate module



boredom

    When there are too many jobs submitted by make, then we have to
    wait until any slot is available.  Unfortunately there is no
    OS-level locking system I can think of that allows us to block
    waiting for any one of a number of resources.

    If there are no slots to run, then at the moment we just sleep for
    2s.  This is OK, but can leave the processor idle.  It would be
    better to be woken up by other processes as they exit.  One way to
    do this would be to listen on a named pipe for notifications.

    This must be backed up by a sleep timer because we may not get the
    notification if e.g. the other process is killed.  Also it won't
    work on Cygwin, which doesn't have named pipes.

    Simply doing a select() on a pipe allows us to block for a while
    or until signalled.  Simply doing a nonblocking write of one byte
    to the pipe ought to allow waking up exactly one of the sleepers.

    Using an OS level semaphore to guard access to slots might work
    with some fudging, but there is no good portable implementation of
    them so it is moot.
    
    When woken, the clients can do one full round of trying to get a
    slot and then go back to sleep.

    This "guides" the OS scheduler towards keeping (almost) the exact
    number of clients activated, without too many of them spinning.

    We can't make the timeout too high, or the client will idle for a
    long time waiting for it.  But if we make it too low then we have
    the thundering herd problem that currently exists...

    Perhaps this is overengineering: people shouldn't make the -j
    number so high that this is hit very often, and we need to have
    the timeout anyhow, so why not just rely on it.

    Just listening on a pipe is cheaper than checking all the locks.



intel CC

    Does not understand the .ii extension.

    We need to specify -xc++ to make it properly compile C++ from
    preprocessed source.

    Is it OK to just get the user to add this?

    Perhaps we could add it always?
    
    Do we need a DISTCC_ADD_OPTIONS variable?


clean up temp files when a client is signalled

    Interrupting a compilation is pretty common.  It might be good to
    handle this more cleanly.

    We can also remove status files.  This would reduce the need for
    monitor clients to handle dead state files, which might reduce
    problems to do with viewing compilations by another user.


globally visible status files
    
    Perhaps store in a world-writable /var/lib/distcc, so that
    they're visible even when TMPDIR or HOME has been reset, as when
    building with emerge.

    Another good case to support is compilation from inside a chroot
    jail.
    
    It might also be nice to be able to see other people using your
    machine either as a client or as a server.

    This requires passing a trust boundary when publishing information
    across accounts.  The directory needs to be writable and the
    programs need to be robust against other users trying to cause
    mischief.  

    It's perhaps not great to allow that kind of security issue in a
    default installation.  Should we really create a mode 777