readme

#ifdef INTEGER #include "ibp.h" #else #include "rbp.h" #endif /* built-in C functions */

save training and testing patterns since normally you would have them
in a file of their own.

   To get a short online tutorial on how to use the program you can type
T at the command prompt and get the listing:

   A Tutorial

   The following topics are designed to be read in the order listed.

   To get help on a topic type the code on the right at the prompt.

   Understanding the Menus                                         h1
   Formatting Data for a Classification Problem                    h2
   Formatting Data for Function Approximation                      h3
   Formatting Data for a Recurrent Problem                         h4
   Making a Network for Classification or Function Approximation   h5
   Making a Recurrent Network                                      h6
   Reading the Data                                                h7
   Setting Algorithms and Parameters                               h8
   Running the Program                                             h9
   Saving Almost Everything                                        h10
   To Quit the Program                                             h11

   To end the program the `q' (for quit) command is entered:

[ACDFGMNPQTW?!acdefhlmopqrstw]? q


4. Basic Facilities
-------------------
   There are a very large number of parameters that can be set for
various algorithms in these programs.  Typing a `?'  will get a compact
listing of them all however they are packed rather tight.  To get a
better view of the parameters there are now many upper-case letter
commands that give a listing of parameters in a less compact form.
These screens list parameters, generally on the left of the screen in
the form of the commands you would need to set them.  The center of the
screen gives a short description of the parameter.  Sometimes one or two
lines are inadequate to describe the command so at the far right there
may be a sequence you can type to get more help with the command.

   The most important screen you can look at is the C for commands
screen that summarizes what each menu screen will show:

[ACDFGMNPQTW?!acdefhlmopqrstw]? C

Screen     Includes Information and Parameters on:

  A        algorithm parameters and tolerance
  C        this listing of major command groups
  D        delta-bar-delta parameters
  F        formats: patterns, output, paging, copying screen i/o
  G        gradient descent (plain backpropagation)
  M        miscellaneous commands: shell escape, seed values, clear,
           clear and initialize, quit, kick a network, run command,
           save almost everything
  N        network building: making a network, initializing
           a network, kicking a network
  P        pattern commands: reading patterns, testing patterns,
  Q        quickprop parameters
  T        a short tutorial
  W        weight commands: listing, saving, restoring
  ?        a compact listing of everything

One typical menu screen is the A screen that lists the main algorithm
parameters:

[ACDFGMNPQTW?!acdefhlmopqrstw]? A

Algorithm Parameters

a a <char>     sets all act. functions to <char>; {ls}             h aa
a ah s         hidden layer(s) act. function; {ls}                 h aa
a ao s         output layer act. function; {ls}                    h aa
a d d          the output layer derivative term; {cdf}             h ad
a i -          initializes units before using the training set; {+-}
a u p          the weight update algorithm; {Ccdpq}                h au
t 0.100        tolerance/unit for successful learning; (0..1)

f O -          allows out-of-date statistics to print; {+-}
f u -          compute up-to-date statistics; {+-}

The first of these listings is the line:

a a <char>     sets all act. functions to <char>; {ls}             h aa

which doesn't give a parameter value but instead it gives the pattern of
a command designed to set the activation function for the entire
network.  The first sequence is:

a a <char>

and this sequence will change the activation function but when you type
it in you will have to substitute a character code for the activation
function instead of the string <char>.  One other activation function is
the linear activation function denoted by the character l, so to get
this function you can type in the line:

a a l

The first `a' codes for the "algorithm" command, the second `a' codes
for the "activation function" and s is the letter for the function.  The
idea of putting the variable portion of the command within the angle
brackets (<>) is a notation devised by Computer Scientists to describe
computer languages.  The word inside these brackets describes the kind
of thing that is the variable portion of the command.

   The middle part of the line:

a a <char>     sets all act. functions to <char>; {ls}             h aa

gives a short description of the meaning of the command and within the
curly brackets there is a listing all the values for all the activation
functions, {ls}.  To get a more detailed explanation of the options type
the sequence on the right: `h aa', this gives:

and the following comes up:

   a a <char> sets every activation function to <char>.
   a ah <char> sets the hidden layer activation function to <char>.
   a ao <char> sets the output layer activation function to <char>,
   <char> can be any of the following:

   <char>                  Function                           Range

      l    linear function, x                             (-inf..+inf)
      s    standard sigmoid, 1 / (1 + exp(-x))               (0..+1)

Here you get the code for the function, the function and the range of
values the function can take on.  This range portion following another
standard of notation used by Mathematicians.  A ( or ) next to a number,
say 0, means the range runs very close to 0 but never exactly to 0, in
other cases (not shown above) a [ or ] next to a number means value can
range up to exactly the number.  Thus the range:

(0..+1]

meaning that the range can run from ALMOST EXACTLY 0 up to exactly +1.

   If we now return to the A screen, the second line was:

a ah s         hidden layer(s) act. function; {ls}                 h aa

Here the idea is to indicate that the activation function for the hidden
layer (or layers) of the network IS NOW the s (standard sigmoid
function).  Again there is a short explanation of this, the set of codes
for functions and information about how to get more help.  This line can
also be taken as a direction as to how to set the hidden layer
activation function as well.  To change it to l you can type in:

a ah l

(Note: normally you would only use the linear activation function in the
output layer of a network.)

   The third line:

a ao l         output layer act. function; {ls}                    h aa

is similar except it states that the activation function for the output
layer is l.


Paging

   In the student version the paging was a simple version of the System
V utility, pg.  Now the paging is more like the common UNIX more
command.  The default page size is 24 lines and it can be reset to
another value with the format command's paging size sub-command.  For
instance to get 12 lines / page instead of 24, use:

f P 12

To get no paging at all use:

f P 0

When the page is full you get the prompt:

More?

At this point you can type:

   q                  to quit viewing the text if you are in a loop,
   a blank            to get another page,
   ^D                 to get another half a page,
   a carriage return  to get one more line and
   c                  to continue without paging.

Mostly paging is needed for loops within the program, like running a
large number of iterations and printing the results, listing the values
of all the patterns or listing weights leading into a particular unit.
Typing the q quits these loops, however paging can also occur with some
of the longer screen menus that are generating lines of output without
running a loop.  For these cases the q does not work.

   Every new command entered from the keyboard sets the page counting
variable to 0 however if input is being taken from a file other than
stdin the counter is NOT reset.  Most of the time this doesn't matter
since the little data files like the xor example used to set up
parameters don't produce any output anyway, however if they do paging is
in effect.  Having paging here is helpful in case there is a problem
with reading the files.

Interrupts

   In UNIX entering an interrupt will stop the current command and the
program will give the user another prompt.  With DOS entering a ctrl-C
will generate a similar kind of interrupt however DOS only checks for
this condition when it has to do i/o.  However when the DOS version is
in a training loop the program also checks to see if a key has been hit
and if that key is the the escape key, the program will break the
training loop.

Control Command

   One control-key command is available in this version, hitting ctrl-R
will run the training algorithm, it is a shorthand for typing r followed
by a carriage return.

Passing Commands to the Operating System

   By using the '!' command you can pass commands to the operating
system from within the program.  The kind of typical things you might
want to do are to list the contents of a directory, list a file or after
saving weights to a file you might want to list them or even edit them
and read them back in.  Here is what you can say for DOS to list the
little data file xor:

! type xor

Once a string has been defined with a ! command it can be re-run simply
by typing the ! followed immediately by a carriage return.

Making a Copy of Your Session

   Sometimes you may want to make of copy of everything that you type in
and the program prints out.  For instance you may get exceptionally good
or bad results using a certain training sequence and an exact record of
what you and the program did could be worth having.  Or you may need an
exact copy of the training or testing set values.  Or you may need lots
of runs where you average the results using another program.  To turn on
the making of a copy use the format command to turn on the copying
process:

f c+

and to turn it off use:

f c-

The text is written to the file copy.

An Alphabetical Listing of the Commands

   The following listing is designed to give you an idea of the set of
commands available.  Details are given in later sections.

a <number>       sets the momentum parameter, alpha
a <options>      the algorithm command
c                clear the network
ci               clear and initialize the network
d <options>      set delta-bar-delta parameters
e <number(s)>    set the learning rate eta
f <options>      lots of formatting options
h <string>       gives more help with certain options
l <layer>        list the values of units on that layer
m <numbers>      make a network
o <number>       list the output targets of the training set pattern
p <options>      list information about training patterns
q                quit
qp               set quickprop parameters
r <options>      run the training algorithm
rt <options>     read the training set patterns
rw <filename>    read the weights
rx <filename>    read the extra training set patterns
s <seeds>        seed value
sb <real>        set bias weights
se <filename>    save almost everything
sw <filename>    save weights
t <options>      list testing file statistics of various sorts
t <real>         tolerance per output unit that must be met
tf <filename>    gives the file name with testing patterns
tr <int>         special test for a recurrent network
trp <int>        special test for a recurrent network
w <layer> <unit> list weights leading into unit

The Summary Line

   The default setting for the summaries you get produce up to
date statistics on the error and on how many patterns are correct.
Here are several lines of summaries from a problem that has training
data and test data for a classification problem:

   10      0.00 %  49.04 % 0.47087       0.00 %  62.50 % 0.38234 
   20      0.00 %  73.08 % 0.38584       0.00 %  77.88 % 0.38108 
   30      2.88 %  76.92 % 0.35043       4.81 %  79.81 % 0.33285 

The first column is of course the number of iterations, the next column
gives the percentage of training patterns that are correct based on the
tolerance.  The next column gives the percentage of correct training
patterns based on the maximum value of the output units.  The next
column is the average absolute value of the error per output unit.  Note
that many other programs will report the RMS error.  The columns on the
right list the percentage of test set patterns that are correct based on
tolerance, the percentage correct based on maximum value and finally the
average error on the test set.