http:^^www.cs.cornell.edu^info^people^lnt^multimatlab.html

来自「This data set contains WWW-pages collect」· HTML 代码 · 共 1,159 行 · 第 1/5 页

[<!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><!WA11><A HREF=#ref_mpibook>7</A>,
<A HREF=#ref_mpi>13</A>].
Here is a brief description of how the system is put together.
<P>

The system is written using 
<!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><!WA12><A HREF=http://www.mcs.anl.gov/Projects/mpi/implementations.html>MPICH</A>,
a popular and freely available implementation of
MPI developed at Argonne National Laboratory and Mississippi State University
[<!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><!WA13><A HREF=#ref_mpich>6</A>].
In particular, MultiMATLAB
uses the P4 communication layer within MPICH, allowing it to run over a
heterogeneous network of workstations. In building upon MPICH, we believe
we have developed a portable and extensible system, in that anyone
can freely get a copy of the software and it will run on many systems.
Versions of MPICH are beginning to become available that run on PCs
running Windows, and we expect soon to experiment with
MultiMATLAB on those platforms.
<P>

The MultiMATLAB <CODE>Start</CODE> command builds a P4 process group file
of remote hosts, which are either explicitly specified
by the user or taken from a default list, and then initializes MPICH.
MATLAB processes are then started on the remote
hosts.  Each process iterates over a simple loop, waiting for and
executing commands received from the user's interactive
MATLAB process.  The user may use a <CODE>Quit</CODE> command to shut down the
slaves and exit MultiMATLAB.  Additionally, if the user quits MATLAB
during a MultiMATLAB session, the slaves are automatically shut down.
<P>

One limitation of MPI, which was not designed for this
particular kind of interactive
use, is that a running program cannot spawn additional processes.
A consequence of this limitation is that once MultiMATLAB is running
on multiple processors, it is not possible to add further processors to the
list except by quitting and starting again.
It is expected that this limitation of MPI will be removed in
the extension of MPI under development known as
<!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><!WA14><A HREF=http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2.html>MPI 2</A>.
<P>

At the user level, MultiMATLAB consists of a collection of
commands such as <CODE>Send</CODE>, for example.  Such a command
is written as a C file called <CODE>Send.c</CODE>, which is interfaced
to MATLAB via the standard
MATLAB Fortran/C/C++ interface system known as MEX.
Within MPI, many variants on sends and receives are defined.
MultiMATLAB is currently built upon the standard send and receive variants,
which employ buffered communication for most messages and synchronous
communication for very large ones.  Our underlying MPI sends and receives
are both blocking operations, to ensure that no data is overwritten,
but to the MultiMATLAB programmer,
the semantics is that <CODE>Recv</CODE> is blocking
while <CODE>Send</CODE> is non-blocking.
<P>

Higher-level MultiMATLAB commands are usually built on
higher-level MPI commands.
For example,
<CODE>Bcast</CODE> and
<CODE>Min</CODE> and
<CODE>Max</CODE> and
<CODE>Sum</CODE> are built on MPI collective communication routines,
and <CODE>Grid</CODE>
and <CODE>Coord</CODE> are built on MPI routines that support
cartesian topologies.
<P>

It should be stressed that MultiMATLAB allows MPI
routines direct access to MATLAB data.
As a result, MultiMATLAB does not incur any extra copying
costs over MPICH, so it is reasonable to expect that its
efficiency should be comparable.  Our experiments show
that this is indeed approximately the case.  Here are the
results of a typical experiment:

<BLOCKQUOTE><CODE><PRE>
     size of matrix      round-trip latency
     (# of doubles)        (milliseconds)
                        MPICH    MultiMATLAB       
       
            25            2.5            4.7
            50            2.1            6.7
           100            2.8           12.6
           200            4.4           15.1
           400            9.3           20.0
           800           18.2           21.1
          1600           35.8           38.4
          3200           80.8           81.9
          6400          165.8          175.7
         12800          339.6          360.8
         25600          708.9          698.7
         51200         1397.4         1406.0
        102400         2744.7         2850.3
</PRE></CODE></BLOCKQUOTE>
     
The table compares round-trip latencies for a MultiMATLAB code with
those for an equivalent C code using MPICH, and reveals
that MultiMATLAB does add some overhead to that of MPICH.
The timings were obtained on the IBM SP2, not using the
high-performance switch.
This occurs because MATLAB performs memory allocation for
received matrices.  It might be possible to alleviate
this problem by maintaining a list of preallocated buffers, but
we have not pursued this idea.

<P><BR><P>
<H2>5. Related Work</H2>

Many people must have thought about parallelizing MATLAB over the
years.  According to Moler's essay "Why there isn't
a parallel MATLAB," published in the MathWorks Newsletter
in 1995 [<!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><!WA15><A HREF=#ref_moler>14</A>],
he was involved with one of the earliest such
attempts in the mid-1980s on an Intel iPSC.  Of course, a great
deal has happened in distributed computing since then.
<P>

Our own first experiments were carried out in
1993 (A. E. Trefethen).  By making use of a Fortran wrapper based
on IBM's message passing environment (MPL), we ran MATLAB on
multiple nodes of an IBM SP-1.  We were impressed
with the power of this system for certain fluid mechanics
calculations, and this experience ultimately led to our persuading
The MathWorks to support us in initiating the present project.
<P>

We are aware of seven projects than have been undertaken elsewhere
that share some of the goals and capabilities of MultiMATLAB.
We shall briefly describe them.
<P>

The longest-standing related project, dating
to before 1990, is the
CONLAB (CONcurrent LABoratory) system of K&aring;gstr&ouml;m
and others at the University of Ume&aring;, Sweden
[<!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><!WA16><A HREF=#ref_conlab>4</A>,<A HREF=#ref_conlab2>10</A>].
CONLAB is a fully-independent system with MATLAB-like
notation that extends the MATLAB language with control
structures and functions for explicit parallelism.
CONLAB programs are compiled into C code with a
message passing library, PICL [<!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><!WA17><A HREF=#ref_picl>5</A>], and
the node computations are done using LAPACK.

<P>

A group at the Center for Supercomputing Research and Development
at the University of Illinois has developed
<!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><!WA18><A HREF=http://www.csrd.uiuc.edu/falcon/falcon.html>FALCON</A>
(FAst Array Language COmputatioN), a programming
environment that facilitates the translation of MATLAB code into
Fortran 90 [<!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><!WA19><A HREF=#ref_falcon>2</A>,<A HREF=#ref_falcon2>3</A>].
FALCON employs compile time and run time inference
mechanisms to determine variable properties such as type, structure,
and size.  Although FALCON does not directly generate parallel code,
the future aim of this project is to annotate the generated Fortran 90
code with directives for parallelization and data distribution.  A
parallelizing Fortran compiler such as
<!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><!WA20><A HREF=http://www.csrd.uiuc.edu/polaris/polaris.html>Polaris</A>
[<!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><!WA21><A HREF=#ref_polaris>1</A>]
may then use these directives to generate parallel code.
<P>
Another project, from the Technion in Israel,
is <!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><!WA22><A HREF=http://techunix.technion.ac.il/~yak/matcom.html>MATCOM</A>
[<!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><!WA23><A HREF=#ref_matcom>12</A>].
MATCOM consists of a MATLAB-to-C++ translator and an
associated C++ matrix class with overloaded operators.
At present, MATCOM
translates MATLAB only into serial C++, but one might hope to
build a distributed C++ matrix class underneath it which would
adhere to the same interface as the existing matrix class.
<P>

A project known as the Alpha Bridge has been developed
by Alpha Data Parallel Systems, Ltd., in
Edinburgh, Scotland [<!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><!WA24><A HREF=#ref_alpha>11</A>].
Originally, in a system known as the MATLAB-Transputer-Bridge,
this group ran a MATLAB-like language in parallel
on each node of a transputer.
The Alpha Bridge system is an enhancement of this idea in which
high-performance RISC processors are linked in a transputer network.
A reduced, MATLAB-like interpreter runs on each node of the network
under the control of a master MATLAB 4.0 process running on a PC.
<P>

A fifth project has been undertaken not far from Cornell
at Integrated Sensors, Inc. (ISI) in Utica, NY, a
consulting company with close links to the US Air Force
Rome Laboratories [<!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><!WA25><A HREF=#ref_isi>9</A>].  Here MATLAB
code is translated to C code with parallel library routines.
This project (and product) aims at executing MATLAB-style programs
in parallel for real-time control and related applications.
<P>

The final two projects we shall mention, though not the most
fully developed, are the closest to our own in concept.
One is a system built by a group at the Universities of Rostock
and Wismar in Germany
[<!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><!WA26><A HREF=#ref_rostock>15</A>,<A HREF=#ref_rostock>16</A>].
In this system MATLAB is run
on various nodes of a network of Unix workstations, with message
passing communication via
the authors' own system PSI/IPC based on Unix sockets.
<P>

Finally, the
<!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><!WA27><A HREF=http://www.mthcsc.wfu.edu/pt/pt.html>Parallel Toolbox</A>
is a system developed originally
by graduate students Pauca, Liu,
Hollingsworth, and Martinez at Wake Forest University in North
Carolina [<!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><!WA28><A HREF=#ref_pt>8</A>].
This system is based upon the message passing system known as
<!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><!WA29><A HREF=http://www.epm.ornl.gov/pvm>PVM</A>.
In the Parallel Toolbox, there is a level of indirection not present
in MultiMATLAB between
the MATLAB master process and the slaves, a PVM
process known as the PT Engine Daemon.  Besides handling
the spawning of new processes, the PT Engine Daemon also filters
input and output, sending error codes to a PT Error Daemon that
logs the error messages to a file.
<P>

In summarizing these various projects, the main thing to be said
is that most of them involve original implementations of a MATLAB-like
language rather than the use of the existing MATLAB system itself.
There are good reasons for this, if one's aim is high performance
and an investigation of what the "ideal" parallel MATLAB-like
system might look like.
The disadvantage is that the existing MATLAB product is
at present so widely used, and so extensive in its
capabilities, that it may be unrealistic and inefficient