paper.tex

国外免费地震资料处理软件包

\title{Reproducible computational experiments \\ using SCons}

\lefthead{Fomel \& Hennenfent}
\righthead{Reproducible research}

\author{Sergey Fomel\/\footnotemark[1] and Gilles Hennenfent\/\footnotemark[2]}

\footnotetext[1]{University of Texas at Austin, E-mail: [e-mail]}
\footnotetext[2]{Earth \& Ocean Sciences, University of British Columbia, E-mail: [e-mail]}

\maketitle

\begin{abstract}
  SCons (from Software Construction) is a well-known open-source
  program designed primarily for building software. In this paper, we
  describe our method of extending SCons for managing data processing
  flows and reproducible computational experiments. We demonstrate our
  usage of SCons with a couple of simple examples.
\end{abstract}

\section{Introduction}

This paper introduces an environment for reproducible computational
experiments developed as part of the ``Madagascar'' software package.
To reproduce the example experiments in this paper, you can download
Madagascar from \url{http://rsf.sourceforge.net/}. At the moment, the
main Madagascar interface is the Unix shell command line so that you
will need a Unix/POSIX system (Linux, Mac OS X, Solaris, etc.) or Unix
emulation under Windows (Cygwin, SFU, etc.)

Our focus, however, is not only on particular
tools we use in our research but also on the general philosophy of
reproducible computations.

\subsection{Reproducible research philosophy}

Peer review is the backbone of scientific progress. From the ancient
alchemists, who worked in secret on magic solutions to insolvable
problems, the modern science has come a long way to become a social
enterprise, where hypotheses, theories, and experimental results are
openly published and verified by the community. By reproducing and
verifying previously published research, a researcher can take new
steps to advance the progress of science.

Traditionally, scientific disciplines are divided into theoretical and
experimental studies. Reproduction and verification of theoretical
results usually requires only imagination (apart from pencils and
paper), experimental results are verified in laboratories using
equipment and materials similar to those described in the publication.

During the last century, computational studies emerged as a new
scientific discipline. Computational experiments are carried out on a
computer by applying numerical algorithms to digital data. How
reproducible are such experiments? On one hand, reproducing the result
of a numerical experiment is a difficult undertaking. The reader needs
to have access to precisely the same kind of input data, software and
hardware as the author of the publication in order to reproduce the
published result. It is often difficult or impossible to provide
detailed specifications for these components. On the other hand, basic
computational system components such as operating systems and
file formats are getting increasingly standardized, and new components
can be shared in principle because they simply represent digital
information transferable over the Internet.

The practice of software sharing has fueled the miraculously efficient
development of Linux, Apache, and many other open-source software
projects.  Its proponents often refer to this ideology as an analog of
the scientific peer review tradition. Eric Raymond, a well-known
open-source advocate, writes \cite[]{taoup}:
\begin{quote}
  Abandoning the habit of secrecy in favor of process transparency and
  peer review was the crucial step by which alchemy became chemistry.
  In the same way, it is beginning to appear that open-source
  development may signal the long-awaited maturation of software
  development as a discipline.
\end{quote}
While software development is trying to imitate science, computational
science needs to borrow from the open-source model in order to sustain
itself as a fully scientific discipline. In words of Randy LeVeque, a
prominent mathematician \cite[]{randy},
\begin{quote}
  Within the world of science, computation is now rightly seen as a
  third vertex of a triangle complementing experiment and
  theory. However, as it is now often practiced, one can make a good
  case that computing is the last refuge of the scientific scoundrel
  [...]  Where else in science can one get away with publishing
  observations that are claimed to prove a theory or illustrate the
  success of a technique without having to give a careful description of
  the methods used, in sufficient detail that others can attempt to
  repeat the experiment? [...]  Scientific and mathematical journals are
  filled with pretty pictures these days of computational experiments
  that the reader has no hope of repeating. Even brilliant and well
  intentioned computational scientists often do a poor job of presenting
  their work in a reproducible manner. The methods are often very
  vaguely defined, and even if they are carefully defined, they would
  normally have to be implemented from scratch by the reader in order to
  test them.
\end{quote}

In computer science, the concept of publishing and explaining computer
programs goes back to the idea of \emph{literate programming} promoted
by \cite{knuth} and expended by many other researchers
\cite[]{thimbleby}. In his 2004 lecture on ``better programming'',
Harold Thimbleby notes\footnote{\url{http://www.uclic.ucl.ac.uk/harold/}}
\begin{quote}
  We want ideas, and in particular programs, that work in one place to
  work elsewhere. One form of objectivity is that published science
  must work elsewhere than just in the author's laboratory or even
  just in the author's imagination; this requirement is called
  \emph{reproducibility}.
\end{quote}

\begin{comment}
The quest for peer review and reproducibility is especially important
for computational geosciences and computational geophysics in
particular. The very first paper published in \emph{Geophysics} was
titled ``Black magic in geophysical prospecting''
\cite[]{GEO01-01-00010008,TLE02-03-00280031} and presented an account
of different ``magical'' methods of oil explorations promoted by
entrepreneurs in the early days of geophysical exploration industry.
Although none of these methods exist today, it is not a secret that
industrial practice is full of nearly magical tricks, often hidden
besides a scientific appearance. Only a scrutiny of peer review and
result verification can help us distinguish magic from science and
advance the latter.
\end{comment}

Nearly ten years ago, the technology of reproducible research in
geophysics was pioneered by Jon Claerbout and his students at the
Stanford Exploration Project (SEP).  SEP's system of reproducible
research requires the author of a publication to document creation of
numerical results from the input data and software sources to let
others test and verify the result reproducibility
\cite[]{SEG-1992-0601,matt}.
The discipline of reproducible research was also adopted and
popularized in the statistics and wavelet theory community by
\cite{donoho}. It is referenced in several popular wavelet theory
books \cite[]{hubbard,mallat}. Pledges for reproducible research
appear nowadays in fields as diverse as bioinformatics
\cite[]{bioconductor}, 
geoinformatics \cite[]{geo}, and computational
wave propagation \cite[]{randy}. However, the adoption or reproducible
research practice by computational scientists has been slow.
Partially, this is caused by difficult and inadequate tools.

\subsection{Tools for reproducible research}

The reproducible research system developed at Stanford is based on
``make \cite[]{make}'', a Unix software construction utility.
Originally, SEP used ``cake'', a dialect of ``make''
\cite[]{Nichols.sep.61.341,Claerbout.sep.67.145,Claerbout.sep.73.451,Claerbout.sep.77.427}.
The system was converted to ``GNU make'', a more standard dialect, by
\cite{Schwab.sep.89.217}.  The ``make'' program keeps track of
dependencies between different components of the system and the
software construction targets, which, in the case of a reproducible
research system, turn into figures and manuscripts. The targets and
commands for their construction are specified by the author in
``makefiles'', which serve as databases for defining source and target
dependencies. A dependency-based system leads to rapid development,
because when one of the sources changes, only parts that depend on
this source get recomputed.  \cite{donoho} based their system on
MATLAB, a popular integrated development environment produced by
MathWorks \cite[]{matlab}.  While MATLAB is an adequate tool for
prototyping numerical algorithms, it may not be sufficient for
large-scale computations typical for many applications in
computational geophysics.

``Make'' is an extremely useful utility employed by thousands of
software development projects. Unfortunately, it is not
well designed from the user experience prospective. ``Make'' employs
an obscure and limited special language (a mixture of Unix shell
commands and special-purpose commands), which often appears confusing
to unexperienced users. According to Peter van der Linden, a software
expert from Sun Microsystems \cite[]{linden},
\begin{quote}
  ``Sendmail'' and ``make'' are two well known programs that are
  pretty widely regarded as originally being debugged into existence.
  That's why their command languages are so poorly thought out and
  difficult to learn. It's not just you -- everyone finds them
  troublesome.
\end{quote}
The inconvenience of ``make'' command language is also in its limited
capabilities.  The reproducible research system developed by
\cite{matt} includes not only custom ``make'' rules but also an
obscure and hardly portable agglomeration of shell and Perl scripts
that extend ``make'' \cite[]{Fomel.sep.94.matt3}.

Several alternative systems for dependency-checking software
construction have been developed in recent years. One of the most
promising new tools is SCons, enthusiastically endorsed by
\cite{dubois}. The SCons initial design won the Software Carpentry
competition sponsored by Los Alamos National Laboratory in 2000 in the
category of ``a dependency management tool to replace make''.  Some of
the main advantages of SCons are:
\begin{itemize}
\item SCons configuration files are Python scripts. Python is a modern
  programming language praised for its readability, elegance,
  simplicity, and power \cite[]{python1,python2}.
  \cite{TLE21-03-02600267} recommend Python as the first programming
  language for geophysics students.
\item SCons offers reliable, automatic, and extensible dependency
  analysis and creates a global view of all dependencies -- no more
  ``make depend'', ``make clean'', or multiple build passes of
  touching and reordering targets to get all of the dependencies.
\item SCons has built-in support for many programming languages and
  systems: C, C++, Fortran, Java, LaTeX, and others.
\item While ``make'' relies on timestamps for detecting file changes
  (creating numerous problems on platforms with different system
  clocks), SCons uses by default a more reliable detection
  mechanism employing MD5 signatures. It can detect changes not only
  in files but also in commands used to build them.
\item SCons provides integrated support for parallel builds.
\item SCons provides configuration support analogous to the ``autoconf''
  utility for testing the environment on different platforms.
\item SCons is designed from the ground up as a cross-platform tool.
  It is known to work equally well on both POSIX systems (Linux, Mac
  OS X, Solaris, etc.) and Windows.
\item The stability of SCons is assured by an incremental development
  methodology utilizing comprehensive regression tests.
\item SCons is publicly released under a liberal open-source license\footnote{As of time of this writing, SCons is in a beta version 0.96 approaching the 1.0 official release. See \url{http://www.scons.org/}.}
\end{itemize}

In this paper, we propose to adopt SCons as a new platform for
reproducible research in scientific computing.

\subsection{Paper organization}

We first give a brief overview of ``Madagascar'' software package and
define the different levels of user interactions. To demonstrate our
adoption of SCons for reproducible research, we then describe a couple
of simple examples of computational experiments and finally show how
SCons helps us document our computational results.

\section{Madagascar software package overview}
%
\inputdir{.}
\plot{rsf_diag}{width=\textwidth}{caption}
%
``Madagascar'' is a multi-layered software package
(Fig.~\ref{fig:rsf_diag}). Users can thus use it in different ways:
%
\begin{itemize}
\item \textbf{command line}: ``Madagascar'' is first of all a
  collection of command line programs. Most programs act as filters on
  input data and can be chained in a Unix pipeline, e.g.
\begin{verbatim}sfspike n1=200 n2=50 | sfnoise rep=y >noise.rsf\end{verbatim}\\
Although these programs mainly focus at this point on geophysical
applications, users can use the API (application programmer's
interface) for writing their own software to manipulate Regularly
Sampled Format (RSF) files, ``Madagascar'' file format. The main