node41.html

Design and building parallel program

constructed from several components.  Simply choosing the optimal
distribution for each component may result in different modules using
different data distributions.  For example, one module may output an
array data structure distributed by columns, while another expects its
input to be distributed by rows.  If these two modules are to be
composed, then either the modules themselves must be modified to use
different distributions, or data must be explicitly redistributed as
they are passed from one component to the other.  These different
solutions can have different performance characteristics and
development costs.
<P>
Both performance tuning and program reuse are made easier if modules
<A NAME=5511>&#160;</A>
are designed to be <em> data distribution neutral
 </em>, that is, if
they can deal with a variety of different data distributions.  This
neutrality can be achieved by specifying the distribution of a
particular data structure as a runtime parameter or in the data
structure itself.  For example, the two modules referred to in the
preceding paragraph could be defined to deal with arbitrary
two-dimensional decompositions.  The combined program could then
utilize a decomposition by rows, a decomposuction
<P>
						<tt> other_computation(grid)</tt>      		 ! More local computation
<P>
						<tt> receive(from_reduce, globmax)</tt>		 ! Receive reduced value
<P>
						<tt> if(globmax <b>&lt;</b> threshold) done = true</tt>
<P>
				<tt> enddo</tt>
<P>
		<tt> end</tt>
<P>
</TT></PRE>

<P>
At each step, this code performs some computation before sending a
message requesting a reduction operation, and then performs other
computation before receiving the result.  The other computation can be
overlapped with the communication required for the reduction.
<P>
<BR><HR>
<P>
<A NAME=5631>&#160;</A>
<P>

<H2><A NAME=SECTION02525000000000000000>4.2.5 Design Rules</A></H2>
<P>
We conclude this section with a set of design rules that can be used
to determine how to compose modules and which sorts of interfaces to
design.
<P>
<OL><LI>
Design modules to handle multiple data distributions.  This feature
can increase their reusability.
<P>
<LI>
Incorporate data distribution information in data structures rather
than in module interfaces.  This approach simplifies interfaces and
maximizes opportunities for code reuse.
<P>
<LI>
Use sequential composition when designing for an SPMD programming
system such as HPF or MPI.
<P>
<LI>
Consider sequential composition when program components cannot execute
concurrently or need to share a lot of data.
<P>
<LI>
Consider concurrent composition if program components can execute
concurrently, communication costs are high, and
communication/computation overlap is possible.
<P>
<LI>
Consider parallel composition if memory is at a premium or if
intracomponent communication costs are greater than intercomponent
communication costs.
</OL>
<P>

<BR> <HR><a href="msgs0.htm#2" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#2"><img ALIGN=MIDDLE src="asm_color_tiny.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/asm_color_tiny.gif" alt="[DBPP]"></a>    <A NAME=tex2html2350 HREF="node40.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node40.html"><IMG ALIGN=MIDDLE ALT="previous" SRC="previous_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/previous_motif.gif"></A> <A NAME=tex2html2358 HREF="node42.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node42.html"><IMG ALIGN=MIDDLE ALT="next" SRC="next_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/next_motif.gif"></A> <A NAME=tex2html2356 HREF="node39.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node39.html"><IMG ALIGN=MIDDLE ALT="up" SRC="up_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/up_motif.gif"></A> <A NAME=tex2html2360 HREF="node1.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node1.html"><IMG ALIGN=MIDDLE ALT="contents" SRC="contents_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/contents_motif.gif"></A> <A NAME=tex2html2361 HREF="node133.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node133.html"><IMG ALIGN=MIDDLE ALT="index" SRC="index_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/index_motif.gif"></A> <a href="msgs0.htm#3" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#3"><img ALIGN=MIDDLE src="search_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/search_motif.gif" alt="[Search]"></a>   <BR>
<B> Next:</B> <A NAME=tex2html2359 HREF="node42.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node42.html">4.3 Performance Analysis</A>
<B>Up:</B> <A NAME=tex2html2357 HREF="node39.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node39.html">4 Putting Components Together</A>
<B> Previous:</B> <A NAME=tex2html2351 HREF="node40.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node40.html">4.1 Modular Design Review</A>
<BR><HR><P>
<P><ADDRESS>
<I>&#169 Copyright 1995 by <A href="msgs0.htm#6" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#6">Ian Foster</a></I>
</ADDRESS>
</BODY>