basic.tex

basic.c */ /**//* Project:NeuroBasic, basic package*//**/ /* Survey:This is a simple Basic b-code

\chapter{Simulating Neural Networks}
%***********************************


\section{Simulator Elements}
\index{simulator elements}
%===========================

The two basic elements of NeuroBasic are {\em neuro-objects\/}
\index{neuro-objects} and {\em neuro-functions}. A neuro-object can be
a layer, a pattern file, etc. These are the elements a neural network
is built of. \index{neuro-functions} Neuro-functions can create and
modify neuro-objects.  A neural network is built by creating the
necessary objects and applying corresponding neuro-functions to
them. The following Chapters describe the neuro-objects and
neuro-functions which belong to neural net algorithm. A few functions
are global and independent of a specific algorithm. They are described
in the following.

\begin{nfunction}{release}{nobj}
deletes the neuro-object {\tt nobj} and frees the memory space which
was occupied by the object. {\tt release(ALL\_OBJECTS)} removes all
existing neuro-objects.
\end{nfunction}


\index{load@\texttt  {load()}|textbf}
\begin{fctitem}{load(nobj, "filename")}
loads the matrix in the file {\tt filename} into neuro-object {\tt
nobj}. All neuro-objects are stored as a matrix in the same file
format (see Appendix~\ref{sec_matformat}). The necessary
(fixed-point/floating-point) conversions are carried out
automatically.

Not all neuro-objects can carry out this function. An error message
will be printed if that is the case.
\end{fctitem}


\index{save@\texttt  {save()}|textbf}
\begin{fctitem}{save(nobj, "filename", format)}
saves the contents of the neuro-object {\tt nobj} into the matrix file
{\tt filename}. The parameter {\tt format} indicates the file format
(see Appendix~\ref{sec_matformat}). It can have the following values:
\begin{list}{}{
\setlength{\parsep}{0em}
\setlength{\itemsep}{0em}
\setlength{\leftmargin}{3.5em}
\setlength{\labelwidth}{3em}
\setlength{\labelsep}{0.5em}    % \leftmargin - \labelwidth
\renewcommand{\makelabel}[1]{\tt#1}
}
\item[FLOAT:] use 32-bit floating-point numbers in a binary file
\item[FIXED:] use 8-bit fixed-point numbers
  ($-8.0$\ldots$7\frac{15}{16}$) in a binary file
\item[ASCII:] store as \ASCII\ text file.
\end{list}
The content of an existing file with the same name will be removed
without a warning. The fixed-point format is not recommended for other
data than patterns because, in most cases, the loss of precision is
not tolerable. Not all neuro-objects can carry out this function.
\end{fctitem}


\index{show@\texttt  {show()}|textbf}
\begin{fctitem}{show(nobj)}
displays the content of neuro-object {\tt nobj} on the screen. The
format is object dependent and will be described together with the
respective objects in the following Chapters.

Not all neuro-objects can carry out this function. An error message
will be printed if that is the case.
\end{fctitem}


\index{nget@\texttt  {nget()}|textbf}
\begin{fctitem}{nget(nobj, i, j)}
returns a single element, specified by the indices {\tt i} and {\tt j}
of a neuro-object. The indices are object dependent and will be
described together with the respective objects in the following
Chapters.

Not all neuro-objects can carry out this function. An error message
will be printed if that is the case.
\end{fctitem}


\index{nput@\texttt  {nput()}|textbf}
\begin{fctitem}{nput(nobj, i, j, value)}
like {\tt nget()} but writes {\tt value} into the indicated element.

Not all neuro-objects can carry out this function. An error message
will be printed if that is the case.
\end{fctitem}


\index{randomize@\texttt  {randomize()}|textbf}
\begin{fctitem}{randomize(nobj, range)}
initializes all elements of neuro-object {\tt nobj} with random
floating-point numbers in the range ({\tt -range\ldots +range}).

Not all neuro-objects can carry out this function. An error message
will be printed if that is the case.
\end{fctitem}


\index{random_seed@\texttt  {random\_seed()}|textbf}
\begin{fctitem}{random\_seed(seed)}
This function uses {\tt seed} as the seed for a new sequence of
pseudo-random numbers for the initialization of fully connected weight
sets. Random seeds are integer numbers, the initial seed is 1.
\end{fctitem}


\index{rnd@\texttt  {rnd()}|textbf}
\begin{fctitem}{rnd(range)}
Returns a random integer number in the range from 0 to {\tt range}
(zero included, {\tt range} not included). A new seed can be set with
the function {\tt random\_seed()}.
\end{fctitem}


\index{rseq@\texttt  {rseq()}|textbf}
\begin{fctitem}{rseq(n)}
This function creates random sequences of arbitrary lengths. In such a
sequence, any number in the range [0\ldots sequence {\tt n}$-1$]
appears exactly once. To create a (new) sequence the function {\tt
rseq()} is called with the negative of the size of the sequence. For
example {\tt rseq(-10)} creates a random sequence where each number
from 0 to 9 appears exactly once. To access an element of the
sequence, the same function is called with the element number. For
example {\tt rseq(3)} returns element number 3 of a previously
generated random sequence. This function can be used to present the
patterns of a training set in random order.
\end{fctitem}






\section{Communication With Other Programs}
\index{communication with other programs}
%==========================================

NeuroBasic can cooperate with the operating system or other programs
to extend its capabilities. The following two commands are expecially
helpful in this context.

\begin{itemize}
\item The {\tt fprint} \index{fprint@{\tt fprint}} command allows to
  create \ASCII\ files of any format, which can be read by other
  programs.
\item The {\tt system()} \index{system@{\tt system()}} function
  allows to start other programs, which run parallel (on multi-tasking
  operating systems) with NeuroBasic.
\end{itemize}

\noindent A useful possibility is the visualization of NeuroBasic
output at runtime. The following example program displays the learning
progress (mean squared error) with {\em Gnuplot\/}\footnote{Gnuplot
can be obtained from {\tt prep.ai.mit.edu}, {\tt ftp.wustl.edu} or
{\tt wuarchive.wustl.edu}} \index{Gnuplot} plotting program. The
listing shows a part of the example program on
Page~\pageref{backprop_example} extended with lines 160, 170, 180 and
230 to invoke Gnuplot for the graphics output.

\begin{verbatim}
  100 rem *** main program ***
  120 rem ====================
  130 create_net(100)    
  140 test_net()
  150 print "epoch "; 0; ":", "error = "; mse,class_rate; " %"
  160 system("rm mse.dat")
  170 fprint "mse.dat", 0, mse
  180 system("gnuplot -title 'NeuroBasic Output' plot.cmd &")
  190 for k = 1 to 10
  200 learn_net() 
  210 test_net()  
  220 print "epoch "; k; ":", "error = "; mse,class_rate; " %"
  230 fprint "mse.dat", k, mse
  240 next k
  250 end
  260 ...
\end{verbatim}

\noindent The program prints {\tt k} and {\tt mse} to the file {\tt
mse.dat}. The command {\tt sys\-tem("gnu\-plot...")} starts Gnuplot
which executes the gnuplot command file {\tt plot.cmd}. Gnuplot
updates the graphics every second with the current data in {\tt
mse.dat}. The command {\tt system("rm mse.dat")} on line 150 deletes a
possibly existing old file {\tt mse.dat} from a previous run. The
following is a listing of the plot command fule {\tt plot.cmd}.

\pagebreak[3]
\begin{verbatim}
  set xlabel "Epoch"
  set ylabel "mse"
  set title "Training Progress"
  set nokey
  plot [0:10][0:0.5] 'mse.dat' with lines 1
  pause 1
  reread
\end{verbatim}




\section{Array Object}
\label{sec_array}
\index{array object}
%=========================================

To avoid thousands of system calls by using the {\tt
fprint()} function during long training sessions, it is highly
recommended to store the desired output values in a special array
object. This array object can then be saved at the end of the
training in a two-dimensional matrix. The following functions are
available for this object type:

\begin{nfunction}{new\_array}{width, height, format}
creates a array object and allocates space for {\tt height}
vectors of size {\tt width} and returns its handle. The parameter
{\tt format} defines the resolution. It can have the following values:
\begin{list}{}{
\setlength{\parsep}{0em}
\setlength{\itemsep}{0em}
\setlength{\leftmargin}{3.5em}
\setlength{\labelwidth}{3em}
\setlength{\labelsep}{0.5em}    % \leftmargin - \labelwidth
\renewcommand{\makelabel}[1]{\tt#1}
}
\item[FLOAT:] use 32-bit floating-point numbers
\end{list}
\end{nfunction}

\begin{nfunction}{nput}{nobj, i, j, value}
stores the number {\tt value} into a single element of the array
{\tt nobj}. The parameter {\tt i} indicates the vector number
(starting at 0) and {\tt j} indicates the element number of that
vector (also starting at 0).
\end{nfunction}

\begin{nfunction}{save}{nobj, "filename", format}
saves the array  {\tt nobj} as a two-dimensional matrix into
file {\tt filename}. The parameter {\tt format} indicates the file
format. It can have the following values:
\begin{list}{}{
\setlength{\parsep}{0em}
\setlength{\itemsep}{0em}
\setlength{\leftmargin}{3.5em}
\setlength{\labelwidth}{3em}
\setlength{\labelsep}{0.5em}    % \leftmargin - \labelwidth
\renewcommand{\makelabel}[1]{\tt#1}
}
\item[FLOAT:] use 32-bit floating-point numbers in a binary file
\item[FIXED:] use 8-bit fixed-point numbers
  ($-8.0$\ldots$7\frac{15}{16}$) in a binary file
\item[ASCII:] store as \ASCII\ text file.
\end{list}
Each line of the matrix represents a vector of the array.

The following is an example of a 3x5 array, stored in the \ASCII
format. See also Appendix~\ref{sec_matformat}.

\begin{verbatim}
    .MAT 2 3 5
    1  2  3  4  5
    4  3  2  6  5
    1  6  4  3  9
\end{verbatim}

\noindent The compact fixed-point format is not recommended to be used
because the resolution is normally too low.
\end{nfunction}

\noindent The following example shows the use of an array object:

\begin{verbatim}
  100 rem *** main program ***
  120 rem ====================
  130 result = new_array(2, 10, FLOAT)
  140 for i = 0 to 9
  150 nput(result, i, 0, i)
  160 nput(result, i, 1, i*i)
  170 nput(result, i, 2, i*i*i)
  180 next i
  190 save(result, "result.mat", ASCII)
\end{verbatim}



\section{NeuroBasic and Operating Systems}
\index{operating systems}
%=========================================

Many features of the host operating system, \eg\ \UNIX\ or \MSDOS, on
which NeuroBasic is executed can be used to build powerful units. Some
examples are given in the following.



\subsection{Command Files}
\index{command files|(}
%-------------------------

To execute a series of experiments NeuroBasic can be called with its
input redirected to a command file. Lets assume the following example
file named {\tt test1.cmd}. It loads the example program (from
Page~\pageref{backprop_example}), modifies it to create a hidden layer
of 50 neurons, runs the program, modifies it again to create a hidden
layer with 150 neurons and runs the program again. The command {\tt
!date} outputs the current date and time by calling the \UNIX\ command
{\tt date}.

\begin{verbatim}
  load demo.bas
  130 create_net(50)
  !date
  run
  130 create_net(150)
  run
  quit
\end{verbatim}

\noindent When called with an input redirection 
\index{input redirection} to this file, NeuroBasic will take its
commands from the file rather than from the keyboard. On \UNIX\ and
\MSDOS\ systems the syntax for input redirection is the ``{\tt<}''
character. The output can be redirected \index{output redirection} in
the same way with the character ``{\tt>}''. The following example
starts NeuroBasic taking its input from the file {\tt test1.cmd}
rather than from the keyboard and writing its output to the file {\tt
logfile.log} rather than to the screen.

\begin{verbatim}
  nbasic <test1.cmd >logfile.log &
\end{verbatim}

\noindent This allows to run arbitrary large series of experiments
off-line, \ie with no user interaction. The ampersand ({\tt\&} at the
end of the command (\UNIX\ systems only) executes NeuroBasic as a
separate task not blocking the shell it was started from while
running.

\UNIX\ provides another mechanism, so called {\em here documents},
\index{here document} which allows to put all Basic commands and the
starting of NeuroBasic itself into one script file\footnote{Script
files are related to batch files on MSDOS systems}. 
\index{script files} The following shell script does exactly the same
thing as the previous example. Let's assume its name is {\tt sc}.

\begin{verbatim}
  nbasic >logfile.log <<EOF
  load demo.bas
  120 create_net(50)
  !date
  run
  120 create_net(150)
  run
  quit
\end{verbatim}

\noindent The experiment can then be started by simply typing

\begin{verbatim}
  sc &
\end{verbatim}

\noindent Note that if the shell script is created with an editor its
executable flag must be set befor it can be called. This is done, for
instance, with the command

\begin{verbatim}
  chmod a+x sc
\end{verbatim}




\subsection{Pipes}
\index{pipes}
%-----------------

Pipes are a possibility of \UNIX\, and other operating systems, to
link the output of one program directly to the input of an other
program.  The following \UNIX\ example pipes the output of NeuroBasic
to the program {\tt tee} which copies its input to two places,
the screen and a file. In this case a copy of NeuroBasic's screen
output will be written into the file {\tt session.log}.

\begin{verbatim}
  nbasic | tee session.log
\end{verbatim} 
\index{command files|)}