portals3.lyx

lustre 1.6.5 source code

.
 We now consider this functionality, along with the concept of groups (see
 the discussion under 
\begin_inset Quotes eld
\end_inset 

other changes
\begin_inset Quotes erd
\end_inset 

), to be part of the runtime system, not part of the Portals API.
 
\emph on 
PtlEQCount
\emph default 
 was defined to return the number of events in an event queue.
 Because external operations may lead to new events being added and other
 threads may remove events, the value returned by 
\emph on 
PtlEQCount
\emph default 
 would have to be a hint about the number of events in the event queue.
\layout Section*

Handling small, unexpected messages
\layout Standard

Another set of changes relates to handling small unexpected messages in
 MPI.
 In designing version 3.0, we assumed that each unexpected message would
 be placed in a unique memory descriptor.
 To avoid the need to process a long list of memory descriptors, we moved
 the memory descriptors out of the match list and hung them off of a single
 match list entry.
 In this way, large unexpected messages would only encounter a single 
\begin_inset Quotes eld
\end_inset 

short message
\begin_inset Quotes erd
\end_inset 

 match list entry before encountering the 
\begin_inset Quotes eld
\end_inset 

long message
\begin_inset Quotes erd
\end_inset 

 match list entry.
 Experience with this strategy identified resource management problems with
 this approach.
 In particular, a long sequence of very short (or zero length) messages
 could quickly exhaust the memory descriptors constructed for handling unexpecte
d messages.
 Our new strategy involves the use of several very large memory descriptors
 for small unexpected messages.
 Consecutive unexpected messages will be written into the first of these
 memory descriptors until the memory descriptor fills up.
 When the first of the 
\begin_inset Quotes eld
\end_inset 

small memory
\begin_inset Quotes erd
\end_inset 

 descriptors fills up, it will be unlinked and subsequent short messages
 will be written into the next 
\begin_inset Quotes eld
\end_inset 

short message
\begin_inset Quotes erd
\end_inset 

 memory descriptor.
 In this case, a 
\begin_inset Quotes eld
\end_inset 

short message
\begin_inset Quotes erd
\end_inset 

 memory descriptor will be declared full when it does not have sufficient
 space for the largest small unexpected message.
\layout Standard

This lead to two significant changes.
 First, each match list entry now has a single memory descriptor rather
 than a list of memory descriptors.
 Second, in addition to exceeding the operation threshold, a memory descriptor
 can be unlinked when the local offset exceeds a specified value.
 These changes have lead to several changes in this document: 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{subsec:paddress}

\end_inset 

: 
\begin_deeper 
\layout Enumerate

removed references to the memory descriptor list, 
\layout Enumerate

changed the portals address translation description to indicate that unlinking
 a memory descriptor implies unlinking the associated match list entry--match
 list entries can no longer be unlinked independently from the memory descriptor.
 
\end_deeper 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:meattach}

\end_inset 

: 
\begin_deeper 
\layout Enumerate

removed unlink from argument list, 
\layout Enumerate

removed description of 
\family typewriter 
ptl_unlink
\family default 
 type, 
\layout Enumerate

changed wording of the error condition when the Portal table index already
 has an associated match list.
 
\end_deeper 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:meinsert}

\end_inset 

: removed unlink from argument list.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:md-type}

\end_inset 

: added 
\family typewriter 
max_offset
\family default 
.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:mdattach}

\end_inset 

: 
\begin_deeper 
\layout Enumerate

added description of 
\family typewriter 
ptl_unlink
\family default 
 type, 
\layout Enumerate

removed reference to memory descriptor lists, 
\layout Enumerate

changed wording of the error condition when match list entry already has
 an associated memory descriptor, 
\layout Enumerate

changed the description of the 
\family typewriter 
unlink
\family default 
 argument.
 
\end_deeper 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:md}

\end_inset 

: removed 
\family typewriter 
PtlMDInsert
\family default 
 operation.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:mdbind}

\end_inset 

: removed references to memory descriptor list.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:mdfree}

\end_inset 

: removed reference to memory descriptor list.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:summary}

\end_inset 

: removed references to PtlMDInsert.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:semantics}

\end_inset 

: removed reference to memory descriptor list.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:exmpi}

\end_inset 

: revised the MPI example to reflect the changes to the interface.
 
\layout Standard

Several changes have been made to improve the general documentation of the
 interface.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:handle-type}

\end_inset 

: documented the special value 
\family typewriter 
PTL_EQ_NONE
\family default 
.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:id-type}

\end_inset 

: documented the special value 
\family typewriter 
PTL_ID_ANY
\family default 
.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:mdbind}

\end_inset 

: documented the return value 
\family typewriter 
PTL_INV_EQ
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:mdupdate}

\end_inset 

: clarified the description of the 
\emph on 
PtlMDUpdate
\emph default 
 function.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:implvals}

\end_inset 

: introduced a new section to document the implementation defined values.
 
\layout Enumerate

Section\SpecialChar ~

\begin_inset LatexCommand \ref{sec:summary}

\end_inset 

: modified Table\SpecialChar ~

\begin_inset LatexCommand \ref{tab:oconsts}

\end_inset 

 to indicate where each constant is introduced and where it is used.
 
\layout Section*

Other changes
\layout Subsection*

Implementation defined limits (Section 
\begin_inset LatexCommand \ref{sec:niinit}

\end_inset 

)
\layout Standard

The earlier version provided implementation defined limits for the maximum
 number of match entries, the maximum number of memory descriptors, etc.
 Rather than spanning the entire implementation, these limits are now associated
 with individual network interfaces.
\layout Subsection*

Added User Ids (Section 
\begin_inset LatexCommand \ref{sec:uid}

\end_inset 

)
\layout Standard

Group Ids had been used to simplify access control entries.
 In particular, a process could allow access for all of the processes in
 a group.
 User Ids have been introduced to regain this functionality.
 We use user ids to fill this role.
\layout Subsection*

Removed Group Ids and Rank Ids (Section 
\begin_inset LatexCommand \ref{sec:pid}

\end_inset 

)
\layout Standard

The earlier version of Portals had two forms for addressing processes: <node
 id, process id> and <group id, rank id>.
 A process group was defined as the collection processes created during
 application launch.
 Each process in the group was given a unique rank id in the range 0 to
 
\begin_inset Formula $n-1$
\end_inset 

 where 
\begin_inset Formula $n$
\end_inset 

 was the number of processes in the group.
 We removed groups because they are better handled in the runtime system.
\layout Subsection*

Match lists (Section 
\begin_inset LatexCommand \ref{sec:meattach}

\end_inset 

)
\layout Standard

It is no longer illegal to have an existing match entry when calling PtlMEAttach.
 A position argument was added to the list of arguments supplied to 
\emph on 
PtlMEAttach
\emph default 
 to specify whether the new match entry is prepended or appended to the
 existing list.
 If there is no existing match list, the position argument is ignored.
\layout Subsection*

Unlinking Memory Descriptors (Section 
\begin_inset LatexCommand \ref{sec:md}

\end_inset 

)
\layout Standard

Previously, a memory descriptor could be unlinked if the offset exceeded
 a threshold upon the completion of an operation.
 In this version, the unlinking is delayed until there is a matching operation
 which requires more memory than is currently available in the descriptor.
 In addition to changes in section, this lead to a revision of Figure\SpecialChar ~

\begin_inset LatexCommand \ref{fig:flow}

\end_inset 

.
\layout Subsection*

Split Phase Operations and Events (Section 
\begin_inset LatexCommand \ref{sec:eq}

\end_inset 

)
\layout Standard

Previously, there were five types of events: 
\family typewriter 
PTL_EVENT_PUT
\family default 
, 
\family typewriter 
PTL_EVENT_GET
\family default 
, 
\family typewriter 
PTL_EVENT_REPLY
\family default 
, 
\family typewriter 
PTL_EVENT_SENT
\family default 
, and 
\family typewriter 
PTL_EVENT_ACK.
 
\family default 
The first four of these reflected the completion of potentially long operations.
 We have introduced new event types to reflect the fact that long operations
 have a distinct starting point and a distinct completion point.
 Moreover, the completion may be successful or unsuccessful.
\layout Standard

In addition to providing a mechanism for reporting failure to higher levels
 of software, this split provides an opportunity for for improved ordering
 semantics.
 Previously, if one process intiated two operations (e.g., two put operations)
 on a remote process, these operations were guaranteed to complete in the
 same order that they were initiated.
 Now, we only guarantee that the initiation events are delivered in the
 same order.
 In particular, the operations do not need to complete in the order that
 they were intiated.
\layout Subsection*

Well known proces ids (Section 
\begin_inset LatexCommand \ref{sec:niinit}

\end_inset 

)
\layout Standard

To support the notion of 
\begin_inset Quotes eld
\end_inset 

well known process ids,
\begin_inset Quotes erd
\end_inset 

 we added a process id argument to the arguments for PtlNIInit.
\layout Chapter*

Glossary
\layout Description

API Application Programming Interface.
 A definition of the functions and semantics provided by library of functions.
 
\layout Description

Initiator A 
\emph on 
process
\emph default 
 that initiates a message operation.
 
\layout Description

Message An application-defined unit of data that is exchanged between 
\emph on 
processes
\emph default 
.
 
\layout Description

Message\SpecialChar ~
Operation Either a put operation, which writes data, or a get operation,
 which reads data.
 
\layout Description

Network A network provides point-to-point communication between 
\emph on 
nodes
\emph default 
.
 Internally, a network may provide multiple routes between endpoints (to
 improve fault tolerance or to improve performance characteristics); however,
 multiple paths will not be exposed outside of the network.
 
\layout Description

Node A node is an endpoint in a 
\emph on 
network
\emph default 
.
 Nodes provide processing capabilities and memory.
 A node may provide multiple processors (an SMP node) or it may act as a
 
\emph on 
gateway
\emph default 
 between networks.
 
\layout Description

Process A context of execution.
 A process defines a virtual memory (VM) context.
 This context is not shared with other processes.
 Several threads may share the VM context defined by a process.
 
\layout Description

Target A 
\emph on 
process
\emph default 
 that is acted upon by a message operation.
 
\layout Description

Thread A context of execution that shares a VM context with other threads.
 
\layout Standard


\begin_inset ERT
status Collapsed

\layout Standard

\backslash 
cleardoublepage
\layout Standard

\backslash 
setcounter{page}{1}
\backslash 
pagenumbering{arabic}
\end_inset 


\layout Chapter

Introduction
\begin_inset LatexCommand \label{sec:intro}

\end_inset 


\layout Section

Overview
\layout Standard