node108.html

Design and building parallel program

log files containing time-stamped event records representing
significant occurrences in a program's execution, such as calling a
procedure or sending a message.  Trace records may include
information such as the type of event and the procedure name or
destination task, and can be generated either automatically or under
programmer control.  Figure <A HREF="node108.html#figpttrace" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node108.html#figpttrace">9.1</A> shows an example of
trace records.
<P>
Trace-based approaches support a particularly broad study of program
behavior.  They can be used to examine causal relationships between
communications, to localize sources of idle time, and to identify
temporary hot spots.  For example, an execution trace could be used to
determine that all processors are sending to the same processor at the
same time.  An execution trace can also be postprocessed to obtain
profile, count, and interval timer information; to compute
higher-order statistics such as the means and variances of these
values; and to obtain other data such as mean message queue length in
a message-passing system.
<P>
<A NAME=14344>&#160;</A>
The disadvantages of trace-based approaches stem primarily from the
huge volume of data that can be generated.  Particularly when a
program is executing on large numbers of processors, it is easy to
generate tens, hundreds, or even thousands of megabytes of data.  (For
example, if a 20-byte record is logged for every message on a
128-processor system, then assuming messages are sent at the rate of
one every 10 milliseconds, trace data will be generated at
256 kilobytes per second, or about 1 gigabyte per hour.)  This large
data volume has three unwelcome consequences.  First, the logging of
this data tends to perturb performance, thereby leading to what is
<A NAME=14345>&#160;</A>
called the <em> probe effect
 </em> in which the measuring of
performance data changes their characteristics.  Second, the sheer
volume of data makes postprocessing difficult.  Frequently,
sophisticated analysis is required to extract relevant information.
Third, the programmer, in order to combat the problems caused by
volume, may have to spend considerable effort tuning the data
collection process so that only relevant events are recorded while the
phenomenon of interest is retained.  Tracing then becomes a
<A NAME=14347>&#160;</A>
labor-intensive process.  For these reasons, tracing should be used with
care and only if other data collection techniques are not available
or do not provide sufficient information.
<P>
Many parallel programming tools provide some automatic tracing
capabilities, for example by generating a trace record for every
message generated or received.  These capabilities are invoked by
linking with a specialized version of a communication library and/or
by a runtime flag.  Mechanisms for generating user-defined events may
also be provided.
<P>
In principle, event traces can be interpreted in various
<A NAME=14348>&#160;</A>
ways by using different tools.  A stumbling block here
is a lack of standards for event log records.  One proposed
<A NAME=14349>&#160;</A>
standard is the Pablo Self Describing Data Format (SDDF) designed at the
University of Illinois.  As illustrated in Figure <A HREF="node108.html#figptsddf" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node108.html#figptsddf">9.2</A>,
this associates an integer event type with a record description that
specifies a type and name for each field.
<P>
<P><A NAME=14581>&#160;</A><IMG BORDER=0 ALIGN=BOTTOM ALT="" SRC="img1066.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/img1066.gif">
<BR><STRONG>Figure 9.2:</STRONG>  An example of the Pablo Self Describing Data Format.
The data record <tt> &quot;Procedure Exit Trace&quot;</tt> has an event type of 105
and six data fields, all integers.
<A NAME=figptsddf>&#160;</A><BR>
<P>
<A NAME=14373>&#160;</A>
<P>
<H2><A NAME=SECTION03624000000000000000>9.2.4 Summary of Data Collection Tools</A></H2>
<P>
A broad spectrum of data collection mechanisms can be used to obtain
<A NAME=14375>&#160;</A>
information about parallel program performance.  In general, those
requiring the least programmer intervention are also the least
intrusive and provide the highest-level, least-detailed view of
program behavior; those providing greater detail are progressively
more intrusive and demand more programmer effort.  Hence, for maximum
programmer efficiency, the process of collecting and interpreting
performance data should proceed in a staged manner, as follows.
<P>
<OL><LI>
Use profile and count information to obtain any parameter values
needed to complete performance models.
<P>
<LI>
Measure execution times for a range of processor counts and problem
sizes, and compare these results with values predicted by performance
models.
<P>
<LI>
If observed and modeled performance differ significantly, use profile
and count information to verify the basic assumptions underlying your
model.  For example, check that message counts and volumes match your
predictions, and check for load imbalances and replicated computation
(Section <A HREF="node32.html#secperfpt" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node32.html#secperfpt">3.6</A>).
<P>
<LI>
If there are still unexplained aspects of program performance,
incorporate simple tracing (or enable automatic tracing capabilities),
and study performance on a few processors.  Increase the number of
processors as your understanding improves.
<P>
</OL>
<P>
Of course, the actual path followed to obtain performance data will
also depend on the functionality provided in a particular parallel
programming system.
<P>
<A NAME=14379>&#160;</A>
<P>

<BR> <HR><a href="msgs0.htm#2" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#2"><img ALIGN=MIDDLE src="asm_color_tiny.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/asm_color_tiny.gif" alt="[DBPP]"></a>    <A NAME=tex2html3264 HREF="node107.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node107.html"><IMG ALIGN=MIDDLE ALT="previous" SRC="previous_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/previous_motif.gif"></A> <A NAME=tex2html3272 HREF="node109.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node109.html"><IMG ALIGN=MIDDLE ALT="next" SRC="next_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/next_motif.gif"></A> <A NAME=tex2html3270 HREF="node106.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node106.html"><IMG ALIGN=MIDDLE ALT="up" SRC="up_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/up_motif.gif"></A> <A NAME=tex2html3274 HREF="node1.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node1.html"><IMG ALIGN=MIDDLE ALT="contents" SRC="contents_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/contents_motif.gif"></A> <A NAME=tex2html3275 HREF="node133.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node133.html"><IMG ALIGN=MIDDLE ALT="index" SRC="index_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/index_motif.gif"></A> <a href="msgs0.htm#3" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#3"><img ALIGN=MIDDLE src="search_motif.gif" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/search_motif.gif" alt="[Search]"></a>   <BR>
<B> Next:</B> <A NAME=tex2html3273 HREF="node109.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node109.html">9.3 Data Transformation and Visualization</A>
<B>Up:</B> <A NAME=tex2html3271 HREF="node106.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node106.html">9 Performance Tools</A>
<B> Previous:</B> <A NAME=tex2html3265 HREF="node107.html" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/node107.html">9.1 Performance Analysis</A>
<BR><HR><P>
<P><ADDRESS>
<I>&#169 Copyright 1995 by <A href="msgs0.htm#6" tppabs="http://www.dit.hcmut.edu.vn/books/system/par_anl/tppmsgs/msgs0.htm#6">Ian Foster</a></I>
</ADDRESS>
</BODY>