source-guide.tex

fortran并行计算包

% \documentstyle[11pt,psfig]{article}
\documentclass[11pt]{article}
\hoffset=-.7in
\voffset=-.6in
\textwidth=6.5in
\textheight=8.5in

\begin{document}
\vspace*{-1in}
\thispagestyle{empty}
\begin{center}
ARGONNE NATIONAL LABORATORY \\
9700 South Cass Avenue \\
Argonne, IL 60439
\end{center}
\vskip .5 in

\begin{center}
\rule{1.75in}{.01in} \\
\vspace{.1in}

ANL/MCS-TM-XXX \\

\rule{1.75in}{.01in} \\

\vskip 1.3 in
{\Large\bf A Guide to the ROMIO MPI-IO Implementation } \\
by \\ [2ex]
{\large\it Robert Ross, Robert Latham, and Rajeev Thakur}
\vspace{1in}

Mathematics and Computer Science Division

\bigskip

Technical Memorandum No.\ XXX


% \vspace{1.4in}
% Revised May 2004

\end{center}

\vfill

{\small
\noindent
This work was supported by the Mathematical, Information, and
Computational Sciences Division subprogram of the Office of Advanced
Scientific Computing Research, U.S. Department of Energy, under
Contract W-31-109-Eng-38; and by the Scalable I/O Initiative, a
multiagency project funded by the Defense Advanced Research Projects
Agency (Contract DABT63-94-C-0049), the Department of Energy, the
National Aeronautics and Space Administration, and the National
Science Foundation.}

\newpage


%%  Line Spacing (e.g., \ls{1} for single, \ls{2} for double, even \ls{1.5})
%%

\newcommand{\ls}[1]
   {\dimen0=\fontdimen6\the\font 
    \lineskip=#1\dimen0
    \advance\lineskip.5\fontdimen5\the\font
    \advance\lineskip-\dimen0
    \lineskiplimit=.9\lineskip
    \baselineskip=\lineskip
    \advance\baselineskip\dimen0
    \normallineskip\lineskip
    \normallineskiplimit\lineskiplimit
    \normalbaselineskip\baselineskip
    \ignorespaces
   }
\renewcommand{\baselinestretch}{1}
\newcommand {\ix} {\hspace*{2em}}
\newcommand {\mc} {\multicolumn}


\tableofcontents
\thispagestyle{empty}
\newpage

\pagenumbering{arabic}
\setcounter{page}{1}
\begin{center}
{\bf Users Guide for ROMIO:  A High-Performance,\\[1ex]
Portable MPI-IO Implementation} \\ [2ex]
by \\ [2ex]
{\it Rajeev Thakur, Robert Ross, Ewing Lusk, and William Gropp}

\end{center}
\addcontentsline{toc}{section}{Abstract}
\begin{abstract}
\noindent
ROMIO is a high-performance, portable implementation of MPI-IO (the
I/O chapter in \mbox{MPI-2}).
This document describes the internals of the ROMIO implementation.
\end{abstract}

\section{Introduction}

The ROMIO MPI-IO implementation, originally written by Rajeev Thakur, has been
in existence since XXX.

... Discussion of the evolution of ROMIO ...

Architecturally, ROMIO is broken up into three layers: a layer implementing
the MPI I/O routines in terms of an abstract device for I/O (ADIO), a layer of
common code implementing a subset of the ADIO interface, and a set of storage
system specific functions that complete the ADIO implementation in terms of
that storage type.  These three layers work together to provide I/O support
for MPI applications.

In this document we will discuss the details of the ROMIO implementation,
including the major components, how those components are implemented, and
where those components are located in the ROMIO source tree.

\section{The Directory Structure}

The ROMIO directory structure consists of two main branches, the MPI-IO branch
(mpi-io) and the ADIO branch (adio).  The MPI-IO branch contains code that
implements the functions defined in the MPI-2 specification for I/O, such as
MPI\_File\_open.  These functions are then written in terms of other functions
that provide an abstract interface to I/O resources, the ADIO functions.
There is an additional glue subdirectory in the MPI-IO branch that defines
functions related to the MPI implementation as a whole, such as how to
allocate MPI\_File structures and how to report errors.

Code for the ADIO functions is located under the ADIO branch.  This code is
responsible for performing I/O operations on whatever underlying storage is
available.  There are two categories of directories in this branch.  The first
is the common directory.  This directory contains two distinct types of
source: source that is used by all ADIO implementations and source that is
common across many ADIO implementations.  This distinction will become more
apparent when we discuss file system implementations.

The second category of directory in the ADIO branch is the file system
specific directory (e.g. ad\_ufs, ad\_pvfs2).  These directories provide code
that is specific to a particular file system type and is only built if that
file system type is selected at configure time.

\section{The Configure Process}

... What can be specified, AIO stuff, where romioconf exists, how to add
another Makefile.in into the list.

\section{File System Implementations}

Each file system implementation exists in its own subdirectory under the adio
directory in the source tree.  Each of these subdirectories must contain at
least two files, a Makefile.in (describing how to build the code in the
directory) and a C source file describing the mapping of ADIO operations to C
functions.

The common practice is to name this file based on the name of the ADIO
implementation.  In the ad\_ufs implementation this file is called ad\_ufs.c,
and contains the following:

\begin{verbatim}
struct ADIOI_Fns_struct ADIO_UFS_operations = {
	ADIOI_UFS_Open, /* Open */
	ADIOI_GEN_ReadContig, /* ReadContig */
	ADIOI_GEN_WriteContig, /* WriteContig */
	ADIOI_GEN_ReadStridedColl, /* ReadStridedColl */
	ADIOI_GEN_WriteStridedColl, /* WriteStridedColl */
	ADIOI_GEN_SeekIndividual, /* SeekIndividual */
	ADIOI_GEN_Fcntl, /* Fcntl */
	ADIOI_GEN_SetInfo, /* SetInfo */
	ADIOI_GEN_ReadStrided, /* ReadStrided */
	ADIOI_GEN_WriteStrided, /* WriteStrided */
	ADIOI_GEN_Close, /* Close */
	ADIOI_GEN_IreadContig, /* IreadContig */
	ADIOI_GEN_IwriteContig, /* IwriteContig */
	ADIOI_GEN_IODone, /* ReadDone */
	ADIOI_GEN_IODone, /* WriteDone */
	ADIOI_GEN_IOComplete, /* ReadComplete */
	ADIOI_GEN_IOComplete, /* WriteComplete */
	ADIOI_GEN_IreadStrided, /* IreadStrided */
	ADIOI_GEN_IwriteStrided, /* IwriteStrided */
	ADIOI_GEN_Flush, /* Flush */
	ADIOI_GEN_Resize, /* Resize */
	ADIOI_GEN_Delete, /* Delete */
};
\end{verbatim}

The ADIOI\_Fns\_struct structure is defined in adio/include/adioi.h.  This
structure holds pointers to appropriate functions for a given file system
type.  "Generic" functions, defined in adio/common, are denoted by the
"ADIOI\_GEN" prefix, while file system specific functions use a file system
related prefix.  In this example, the only file system specific function is
ADIOI\_UFS\_Open.  All other operations use the generic versions.

Typically a third file, a header with file system specific defines and
includes, is also provided and named based on the name of the ADIO
implementation (e.g. ad\_ufs.h).

Because the UFS implementation provides its own open function, that code must be provided in the ad\_ufs subdirectory.  That function is implemented in adio/ad\_ufs/ad\_ufs\_open.c.

\section{Generic Functions}

As we saw in the discussion above, generic ADIO function implementations are
used to minimize the amount of code in the ROMIO tree by sharing common
functionality between ADIO implementations.  As the ROMIO implementation has
grown, a few categories of generic implementations have developed.  At this
time, these are all lumped into the adio/common subdirectory together, which
can be confusing.

The easiest category of generic functions to understand is the ones that
implement functionality in terms of some other ADIO function.
ADIOI\_GEN\_ReadStridedColl is a good example of this type of function and is
implemented in adio/common/ad\_read\_coll.c.  This function implements
collective read operations (e.g. MPI\_File\_read\_at\_all).  We will discuss how
it works later in this document, but for the time being it is sufficient to
note that it is written in terms of ADIO ReadStrided or ReadContig calls.

A second category of generic functions are ones that implement functionality
in terms of POSIX I/O calls.  ADIOI\_GEN\_ReadContig (adio/common/ad\_read.c) is
a good example of this type of function.  These "generic" functions are the
result of a large number of ADIO implementations that are largely POSIX I/O
based, such as the UFS, XFS, and PANFS implementations.  We have discussed
moving these functions into a separate common/posix subdirectory and renaming
them with ADIOI\_POSIX prefixes, but this has not been done as of the writing
of this document.

The next category of generic functions holds functions that do not actually
require I/O at all.  ADIOI\_GEN\_SeekIndividual (adio/common/ad\_seek.c) is a
good example of this.  Since we don't need to actually perform I/O at seek
time, we can just update local variables at each process.  In fact, one could
argue that we no longer need the ADIO SeekIndividual function at all - all the
ADIO implementations simply use this generic version (with the exception of
TESTFS, which prints the value as well).

The next category of generic functions are the "FAKE" functions (e.g.
ADIOI\_FAKE\_IODone implemented in adio/common/ad\_done\_fake.c).  These functions
are all related to asynchronous I/O (AIO) operations.  These implement the AIO
operations in terms of blocking operations - in other words, they follow the
standard but do not allow for overlap of I/O and computation or communication.
These are used in cases where AIO support is otherwise unavailable or
unimplemented.

The final category of generic functions are the "na侊ve" functions (e.g.
ADIOI\_GEN\_WriteStrided\_naive in adio/common/ad\_write\_str\_naive.c).  These
functions avoid the use of certain optimizations, such as data sieving.

Other Things in adio/common