users-guide.tex

mpi并行计算的c++代码 可用vc或gcc编译通过 可以用来搭建并行计算试验环境

With SGI MPI, you can compile MPI-IO programs as \\
\hspace*{.4in} {\tt cc foo.c -lmpi}\\
or \\
\hspace*{.4in} {\tt f77 foo.f -lmpi}\\
or \\
\hspace*{.4in} {\tt f90 foo.f -lmpi}\\

With LAM, you can compile MPI-IO programs as \\
\hspace*{.4in} {\tt hcc foo.c -lmpi}\\
or \\
\hspace*{.4in} {\tt hf77 foo.f -lmpi}\\

If you have built ROMIO with some other MPI implementation, you can
compile MPI-IO programs by explicitly giving the path to the include
file mpio.h or mpiof.h and explicitly specifying the path to the
library libmpio.a, which is located in {\tt \$(ROMIO\_HOME)/lib/\$(ARCH)/libmpio.a}.

Run the program as you would run any MPI program on the machine.
If you use {\tt mpirun}, make sure you use the correct {\tt mpirun}
for the MPI implementation you are using. For example, if you
are using MPICH on an SGI machine, make sure that you use MPICH's
{\tt mpirun} and not SGI's {\tt mpirun}.


%
% LIMITATIONS
%
\section{Limitations of This Version of ROMIO \label{sec:limit}}

\begin{itemize}
\item When used with any MPI implementation other than MPICH revision
1.2.1 or later, the {\tt status} argument is not filled in any MPI-IO
function. Consequently, {\tt MPI\_Get\_count} and\linebreak {\tt
MPI\_Get\_elements} will not work when passed the {\tt status} object
from an MPI-IO operation.

\item Additionally, when used with any MPI implementation other than MPICH
revision 1.2.1 or later, all MPI-IO functions return only two possible
error codes---{\tt MPI\_SUCCESS} on success and {\tt MPI\_ERR\_UNKNOWN}
on failure.

\item All nonblocking I/O functions use a ROMIO-defined {\tt MPIO\_Request}
object instead of the usual {\tt MPI\_Request} object. Accordingly, two
functions, {\tt MPIO\_Test} and {\tt MPIO\_Wait}, are provided to test
and wait on these {\tt MPIO\_Request} objects. They have the same
semantics as {\tt MPI\_Test} and {\tt MPI\_Wait}.\\
\hspace*{.4in} {\tt int MPIO\_Test(MPIO\_Request *request, int *flag, MPI\_Status *status);}\\
\hspace*{.4in} {\tt int MPIO\_Wait(MPIO\_Request *request, MPI\_Status *status);}\\
The usual functions {\tt MPI\_Test}, {\tt MPI\_Wait}, {\tt MPI\_Testany},
and so forth, will not work for nonblocking I/O. 

\item This version works only on a homogeneous cluster of machines,
and only the ``native'' file data representation is supported.

\item Shared file pointers are not supported on PVFS and IBM PIOFS
file systems because they don't support {\tt fcntl} file locks,
and ROMIO uses that feature to implement shared file pointers.

\item On HP machines running HPUX and on NEC SX-4, you need to compile
Fortran programs with {\tt mpif90} instead of {\tt mpif77}, because
the {\tt f77} compilers on these machines don't support 8-byte integers.

\item The file-open mode {\tt MPI\_MODE\_EXCL} does not work on Intel
PFS file system, due to a bug in PFS.

\end{itemize}


%
% USAGE TIPS
%
\section{Usage Tips}
\begin{itemize}
\item When using ROMIO with SGI MPI, you may
sometimes get an error message from SGI MPI: ``MPI has run out of
internal datatype entries. Please set the environment variable
{\tt MPI\_TYPE\_MAX} for additional space.'' If you get this error message,
add the following line to your {\tt .cshrc} file:\\
\hspace*{.4in} {\tt setenv MPI\_TYPE\_MAX 65536}\\
Use a larger number if you still get the error message.
\item If a Fortran program uses a file handle created using ROMIO's C
interface, or vice versa, you must use the functions {\tt MPI\_File\_c2f} 
or {\tt MPI\_File\_f2c} (see \S~4.12.4 in~\cite{mpi97a}). Such a
situation occurs, for example, if a Fortran program uses an I/O
library written in C 
with MPI-IO calls. Similar functions {\tt MPIO\_Request\_f2c} and
{\tt MPIO\_Request\_c2f} are also provided.
\item For Fortran programs on the Intel Paragon, you may need
to provide the complete path to {\tt mpif.h} in the {\tt include}
statement, e.g., \\
\hspace*{.4in} {\tt include '/usr/local/mpich/include/mpif.h'}\\
instead of \\
\hspace*{.4in} {\tt include 'mpif.h'}\\ 
This is because the {\tt -I}
option to the Paragon Fortran compiler {\tt if77} doesn't work
correctly. It always looks in the default directories first and,
therefore, picks up Intel's {\tt mpif.h}, which is actually the {\tt
mpif.h} of an older version of MPICH.

\end{itemize}

%
% MAILING LIST
%
\section{ROMIO Users Mailing List}
Please register your copy of ROMIO with us by sending e-mail
to {\tt [e-mail]} with the message

{\tt subscribe romio-users}

{\noindent This} will enable us to notify you of new releases of ROMIO
as well as bug fixes.


%
% REPORTING BUGS
%
\section{Reporting Bugs}
If you have trouble, first check the users guide. Then check if there
is a list of known bugs and patches on the ROMIO web page at {\tt http://www.mcs.anl.gov/romio}.
Finally, if you still have problems, send a detailed message containing:\\
\hspace*{.2in}$\bullet$ the type of system (often {\tt uname -a}),\\
\hspace*{.2in}$\bullet$ the output of {\tt configure},\\
\hspace*{.2in}$\bullet$ the output of {\tt make}, and \\
\hspace*{.2in}$\bullet$ any programs or tests\\
to {\tt [e-mail]}.


%
% ROMIO INTERNALS
%
\section{ROMIO Internals}
A key component of ROMIO that enables such a portable MPI-IO
implementation is an internal abstract I/O device layer called
ADIO~\cite{thak96e}. Most users of ROMIO will not need to deal with
the ADIO layer at all. However, ADIO is useful to those who want to
port ROMIO to some other file system. The ROMIO source code and the
ADIO paper~\cite{thak96e} will help you get started.

MPI-IO implementation issues are discussed in~\cite{thak99b}. All
ROMIO-related papers are available online at {\tt
http://www.mcs.anl.gov/romio}.


\section{Learning MPI-IO}
The book {\em Using MPI-2: Advanced Features of the Message-Passing
Interface}~\cite{grop99a}, published by MIT Press, provides a tutorial
introduction to all aspects of MPI-2, including parallel I/O. It has
lots of example programs. See {\tt
http://www.mcs.anl.gov/mpi/usingmpi2} for further information about
the book.

%
% MAJOR CHANGES IN PREVIOUS RELEASES
%
\section{Major Changes in Previous Releases}

\subsection{Major Changes in Version 1.2.3}
\begin{itemize}
\item Added explicit control over aggregators for collective operations
      (see description of \texttt{cb\_config\_list}).
\item Added the following working hints: \texttt{cb\_config\_list},
      \texttt{romio\_cb\_read}, \texttt{romio\_cb\_write},\newline
      \texttt{romio\_ds\_read}.  These additional hints have
      been added but are currently ignored by the implementation:
      \texttt{romio\_ds\_write}, \texttt{romio\_no\_indep\_rw}.
\item Added NTFS ADIO implementation.
\item Added testfs ADIO implementation for use in debugging.
\item Added delete function to ADIO interface so that file systems that
      need to use their own delete function may do so (e.g. PVFS).
\item Changed version numbering to match version number of MPICH release.
\end{itemize}

\subsection{Major Changes in Version 1.0.3}
\begin{itemize}
\item When used with MPICH 1.2.1, the MPI-IO functions return proper
error codes and classes, and the status object is filled in.

\item On SGI's XFS file system, ROMIO can use direct I/O even if the
user's request does not meet the various restrictions needed to use
direct I/O. ROMIO does this by doing part of the request with buffered
I/O (until all the restrictions are met) and doing the rest with
direct I/O. (This feature hasn't been tested rigorously. Please check
for errors.)

By default, ROMIO will use only buffered I/O. Direct I/O can be
enabled either by setting the environment variables {\tt
MPIO\_DIRECT\_READ} and/or {\tt MPIO\_DIRECT\_WRITE} to {\tt TRUE}, or
on a per-file basis by using the info keys {\tt direct\_read} and {\tt
direct\_write}.

Direct I/O will result in higher performance only if you are accessing
a high-bandwidth disk system. Otherwise, buffered I/O is better and is
therefore used as the default.

\item Miscellaneous bug fixes.
\end{itemize}

\subsection{Major Changes in Version 1.0.2}
\begin{itemize}
\item Implemented the shared file pointer functions (\S~9.4.4 of MPI-2) and 
  split collective I/O functions (\S~9.4.5). Therefore, the main
   components of the MPI-2 I/O chapter not yet implemented are 
  file interoperability and error handling.

\item Added support for using ``direct I/O'' on SGI's XFS file system. 
  Direct I/O is an optional feature of XFS in which data is moved
  directly between the user's buffer and the storage devices, bypassing 
  the file-system cache. This can improve performance significantly on 
  systems with high disk bandwidth. Without high disk bandwidth,
  regular I/O (that uses the file-system cache) perfoms better.
  ROMIO, therefore, does not use direct I/O by default. The user can
  turn on direct I/O (separately for reading and writing) either by
  using environment variables or by using MPI's hints mechanism (info). 
  To use the environment-variables method, do
\begin{verbatim}
       setenv MPIO_DIRECT_READ TRUE
       setenv MPIO_DIRECT_WRITE TRUE
\end{verbatim}
  To use the hints method, the two keys are {\tt direct\_read} and {\tt
  direct\_write}.  By default their values are {\tt false}. To turn on
  direct I/O, set the values to {\tt true}. The environment variables
  have priority over the info keys.  In other words, if the environment
  variables are set to {\tt TRUE}, direct I/O will be used even if the
  info keys say {\tt false}, and vice versa.  Note that direct I/O must be
  turned on separately for reading and writing.  The environment-variables
  method assumes that the environment variables can be read by each
  process in the MPI job. This is not guaranteed by the MPI Standard,
  but it works with SGI's MPI and the {\tt ch\_shmem} device of MPICH.

\item Added support (new ADIO device, {\tt ad\_pvfs}) for the PVFS parallel 
  file system for Linux clusters, developed at Clemson University
  (see {\tt http://www.parl.clemson.edu/pvfs}). To use it, you
  must first install PVFS and then when configuring ROMIO, specify
  {\tt -file\_system=pvfs} in addition to any other options to {\tt
  configure}. (As usual, you can configure for multiple file systems by
  using ``{\tt +}''; for example, {\tt -file\_system=pvfs+ufs+nfs}.) You
  will need to specify the path to the PVFS include files via the {\tt
  -cflags} option to {\tt configure}, for example, \newline {\tt configure
  -cflags=-I/usr/pvfs/include}. You will also need to specify the full
  path name of the PVFS library. The best way to do this is via the {\tt
  -lib} option to MPICH's {\tt configure} script (assuming you are using
  ROMIO from within MPICH).

\item Uses weak symbols (where available) for building the profiling version,
  i.e., the PMPI routines. As a result, the size of the library is reduced
  considerably. 

\item The Makefiles use {\em virtual paths} if supported by the make
  utility. GNU {\tt make}
  supports it, for example. This feature allows you to untar the
  distribution in some directory, say a slow NFS directory,
  and compile the library (create the .o files) in another 
  directory, say on a faster local disk. For example, if the tar file
  has been untarred in an NFS directory called {\tt /home/thakur/romio},
  one can compile it in a different directory, say {\tt /tmp/thakur}, as
  follows:
\begin{verbatim}
        cd /tmp/thakur
        /home/thakur/romio/configure
        make
\end{verbatim}
  The .o files will be created in {\tt /tmp/thakur}; the library will be created in\newline
  {\tt /home/thakur/romio/lib/\$ARCH/libmpio.a}.
  This method works only if the {\tt make} utility supports {\em
  virtual paths}. 
  If the default {\tt make} utility does not, you can install GNU {\tt
  make} which does, and specify it to {\tt configure} as
\begin{verbatim}
       /home/thakur/romio/configure -make=/usr/gnu/bin/gmake (or whatever)
\end{verbatim}

\item Lots of miscellaneous bug fixes and other enhancements.

\item This version is included in MPICH 1.2.0. If you are using MPICH, you
  need not download ROMIO separately; it gets built as part of MPICH.
  The previous version of ROMIO is included in LAM, HP MPI, SGI MPI, and 
  NEC MPI. NEC has also implemented the MPI-IO functions missing 
  in ROMIO, and therefore NEC MPI has a complete implementation of MPI-IO.
\end{itemize}


\subsection{Major Changes in Version 1.0.1}

\begin{itemize}
\item This version is included in MPICH 1.1.1 and HP MPI 1.4.

\item Added support for NEC SX-4 and created a new device {\tt ad\_sfs} for
NEC SFS file system.

\item New devices {\tt ad\_hfs} for HP HFS file system and {\tt
ad\_xfs} for SGI XFS file system.

\item Users no longer need to prefix the filename with the type of 
file system; ROMIO determines the file-system type on its own.

\item Added support for 64-bit file sizes on IBM PIOFS, SGI XFS,
HP HFS, and NEC SFS file systems.

\item {\tt MPI\_Offset} is an 8-byte integer on machines that support
8-byte integers. It is of type {\tt long long} in C and {\tt
integer*8} in Fortran. With a Fortran 90 compiler, you can use either
{\tt integer*8} or  {\tt integer(kind=MPI\_OFFSET\_KIND)}. 
If you {\tt printf} an {\tt MPI\_Offset} in C, remember to use {\tt \%lld} 
or {\tt \%ld} as required by your compiler. (See what is used in the test 
program {\tt romio/test/misc.c}).
On some machines, ROMIO detects at configure time that {\tt long long} is 
either not supported by the C compiler or it doesn't work properly.
In such cases, configure sets {\tt MPI\_Offset} to {\tt long} in C and {\tt
integer} in Fortran. This happens on Intel Paragon, Sun4, and FreeBSD.

\item Added support for passing hints to the implementation via the
{\tt MPI\_Info} parameter. ROMIO understands the following hints (keys
in {\tt MPI\_Info} object):
\texttt{cb\_buffer\_size},
\texttt{cb\_nodes},\newline
\texttt{ind\_rd\_buffer\_size},
\texttt{ind\_wr\_buffer\_size} (on all but IBM PIOFS),
\texttt{striping\_factor} (on PFS and PIOFS),
\texttt{striping\_unit} (on PFS and PIOFS),
\texttt{start\_iodevice} (on PFS and PIOFS),
and \texttt{pfs\_svr\_buf} (on PFS only).
      
\end{itemize}
\newpage

\addcontentsline{toc}{section}{References}
\bibliographystyle{plain}
%\bibliography{/home/beumer/THAKUR/TM234/papers}
\bibliography{/home/thakur/tex/bib/papers}

\end{document}