manual.txt

BP神经网络算法的VC＋源码

hidden2->Setf( -2.0, 2.0, -4.0, 2.0 );
output->Setf( 3.0, 1.0 );
//-------------------------------------------

There are two ways to Set values, Setf and Seti. Setf is for floating point and
double matrices. Seti is for integer matrices.

If you also want to insert bias weights (or weights that always receive an input of 1),
you may do that by typing the following lines after the code you just inserted:

//--------------------------------------------
derived.LoadVectorf( hidden1->weight, 2, 2.0, 2.0 );
derived.LoadVectorf( hidden2->weight, 2, 3,0, -2.0 );
derived.LoadVectorf( output->weight, 1, -2.0 );
//--------------------------------------------

The "weight" vector of each layer holds the biased weights. The LoadVector
functions takes this vector as the first value. The next value is the length
of the vector, and all the following values are the values to insert into
the vector in sequential order. A specific element in a vector may be accessed
by an id: hidden1->weight[5], accesses the 6th element in weight, just like
an array. There are two ways in which to load vectors, either LoadVectorf or
LoadVectori. LoadVectorf is for floating point and double. LoadVectori is
for integer VECTORs.

You have now created a network that is represented in matrix form like this:

bias: 	 [  2  2 ] [  3 -2 ] [ -2 ]

weights: [ -2  3 ] [ -2  2 ] [  3 ]
weights: [ -2  3 ] [ -4  2 ] [  1 ]

If you want to See a printout of this you can simply type the following line
into the main function and run the program:

derived.DisplayLAYER( &ppHidden1 );

Notice that you are again using the handler because "derived" expects a <float>.

Check the reference section in the table of contents to learn how to read and
write layers from input and output streams.

2.3.2 Feed Forward sweep
--------------------------------------------------------------------------

Now, we must define an input, a final product, and a target. To do this
we must use VECTOR(s). Vectors are in a simple definition: matrix rows or columns.
When you present your networks with an input, you will do this in vector form. For
example an input vector may look like this: [  9 8 44 3 2 ]
Define the input, output (FINAL), and target vectors like this:
(from now on simply append this code in the main function under the code
you have just inserted)

//-----------------------------------------
nn_utility_functions<float>::VECTOR input, FINAL, target;
//----------------------------------------

Next, use the LoadVector function to put values into the vectors you have just
created. The first value you will pass to the function will be the vector you
wish to load, the second will be the length of the vector (how many elements you
are planning on inserting), and the next values will be the actual elements.
Type the following to load the vectors with values:

//-----------------------------------------
derived.LoadVector( input, 2, 0.1, 0.9 );
derived.LoadVector( target, 1, 0.9 );
//----------------------------------------

Right now we are not at the point of defining our own functions, but if we were
to define our own function we would have to read section 5.5 "Defining your own
network functions".

Now, if you want to perform a Feed Forward sweep, type the following line in the "main"
function:

//--------------------------------------------------------
hidden1->FeedForward( input, FINAL );
//----------------------------------------------------------

What this does is present the "input" vector to the first layer. Once the Feed Forward
sweep is done and the final layer has presented an output, the output will be stored
in vector format in the FINAL vector.

To view the result of the Feed Forward sweep, type the following line into your program:

//----------------------------------------------------------
derived.PrintVector( FINAL, 1 );
//---------------------------------------------------------

This line simply performs a screen dump of the output vector (FINAL) that is of length 1,
since this is the number of columns in the "output" layer we defined earlier.

2.3.3 Back Propagation sweep
------------------------------------------------------------------------

Now, as you will notice this output is not correct (compared to the target)
so the network must back-propagate. If you are interested in programming your
own update function I suggest taking a look at section 5.5. We have already
declared the function as a SIGMOID function, so we may type the following lines
into the main() function:

//----------------------------------------------------------------
output->BackPropagate( input, target, 0.3, 1 );
//---------------------------------------------------------------

What this does is backpropagate from the "output" layer, by presenting
it with the input and the target that we had previously defined. 0.3 is the
learning rate, and 1 is the length of the target.

If you display the network like we had done previously with the Display command, you will
notice changes in the weight matrix. Moreover, if you Feed Forward again you will notice
that the output is closer to the intended target. The entire source code for this example
can be found in the examples directory. The file is named "simple_sigmoid.cpp".


2.3.4 General IMPORTANT Note
----------------------------------------------------------------
Note that whenever you use a function like LoadVector or CopyVector or train (latter on)
or PrintVector, you must precede the command with a "derived." or any other derivation
of the nn_utility_functions class. For example, you defined "derived" in the latter example
as:

nn_utility_functionsi<int> derived;

You could have done this by defining a class as well:

class MY_DERIVED : public nn_utility_functions<double>{
	public:
		int another_integer;
		//.
		//. more code or functions .....
		//.
}derived;

"derived" now has all the properties of nn_utility_functions, + any other functions
you implemented. This means that whenever you use a function such as PrintVector
or LoadVector, you must precede it with a "derived." or any other name you choose to use.

3.0 Intermediate: Training a Neural Network
---------------------------------------------------

I am going to assume that you have read the Basics section.
This section (3.0) is aimed a teaching you how to use the training
functions.

Make sure that you have defined a network topology and weights
before proceeding.

Output during a training process is somewhat up to you, you may
include cout statements where you see fit, and you may also ask
the training function to print some limited output during the
training process.

3.1 The GetInput Function
-------------------------------------------------------

The first function you will need to define the body for is the "GetInput"
function.  What you need to know follows:

The GetInput function is derived from the nn_utility_functions class. To overload
the default function, which does not do anything, and derive your own function you
must nfirst create a very simple class:

class MY_CLASS : public nn_utility_functions<float>{
public:
	typdef float VECTOR[NN_UTIL_SIZE];

	void GetInput( int iteration, VECTOR &send, VECTOR &target );
}derived;

If you do this, you must remove the line from the top of your source code that
says:

nn_utility_function<float> derived;

Because you redefine this object with your own GetInput function implemented.

Notice the new typedef declaration. This is done because if we used the typedef
declaration of nn_utility_functions this would cause a linker error known as
"implicit typedef". To avoid this we redefine VECTOR as a floating point. If
we also performed operations on MATRIX typedefs, we would have to redefine MATRIX
in the following manner: "typdef float MATRIX[NN_UTIL_SIZE][NN_UTIL_SIZE];".

The GetInput function is called any time a new training pattern is required. It is up
to you to set the value for send and target. An example could be as follows:

void MY_CLASS::GetInput( int iteration, VECTOR &send, VECTOR &target ){
	LoadVector( send, 2, 0.9, 0.1 );
	LoadVector( target, 1, 0.9 );
}

The "derived." preceedor is not required because this code is in the GetInput function
and the GetInput function belongs to a nn_utility_functions derived class, hence it
has inherited the LoadVector function.

The iteration variable is sent to the function and represents the counter
of iterations of training patterns. This means that if you are training the
network for the 500th time, this number will be 500.

If you want to create random patterns to test the network with, you can use
the rand() function and select patterns from a list. You may also want to
train the network using say 4 patterns over and over again, to do this
use a counter that repeats four times and use a select switch:

switch( choose ){
...
}

choose = ( choose == 4 ? 0 : choose+1);

Some good examples of this are in the examples folder: "sigmoid_example.cpp"
and "kohen_example.cpp".

3.2 The CheckTrain Function
---------------------------------------------------------

The next function you must define is the CheckTrain function. You can include
this function definition in the same class as the GetInput function:

bool CheckTrain( VECTOR output, VECTOR target, int length );

This function provides you with 3 variables: the network's output "output",
the target you previously set "target", and the length of both of these "length".
Your task is to return a value "true" if you wish the network to train, or "false"
if you do not deem the training necessary. The default value is to always return true,
so only overload this function if you plan on doing otherwise.

Again, to enter the body code do the following:

bool MY_CLASS::CheckTrain..... just like the GetInput Function

Some good examples of this function can be found in the same programs that
were suggested for the 3.1 section.

3.3 The train Function
-------------------------------------------------------
Finally, in the main function you must begin the training process by using the following function:

derived.train( &ppINPUT_LAYER, &ppOUTPUT_LAYER, INTERATIONS, LEARNING_RATE, OUTPUT_OR_NOT )

For example, suppose you have defined a network with the first hidden layer being named: "hidden1",
and the output layer being named "output", much like the one designed in section 2.0. If you want to
train for 1000 iterations, or epochs, at a learning rate of 0.7, and you want the function to
present output:

derived.train( &ppHidden1, &ppOutput, 1000, 0.7, true );

where ppHidden1 and ppOutput are buffers to hidden1 and output.

if you did not want an output:

derived.train( &ppHidden1, &ppOutput, 1000, 0.7, false );

And that's it, that's all there is to training. Almost all of the examples use training
functions and train the networks they create. Take a look through the directory. Also,
if you have questions post a message in the nn-utility forum or post a support request.
Visit http://sourceforge.net/projects/nn-utility to do either of these. For more information
feel free to contact [e-mail].

4.0 Accessories: Using Bitmaps
-------------------------------------------------

Bitmaps are an accessory I have added to the nn-utility library. If you have any
ideas for accessories please contact me. If you have  programmed any accessories
please notify me so that they may be complied into the library and tested. The newest
accessory addition is a graphing utility that analyzes learning. It is a separate
library that uses OpenGL, it is also simple to use. Visit
http://sourceforge.net/projects/nn-utility to download it. Note that the graph-utility
requires version 1.1 to be compatible with version 1.6 of nn-utility, and v1.3 to
be compatible with version 1.9 of nn-utility.

Now, about using the bitmaps. The format is quite simple, bitmaps can be used
in training or for any other reason you deem necessary. The bitmap class
consists of some basic functions:

firstly the definition:

BITMAP<float> *bit = new BITMAP<float>( "bitmap.txt" ); //define the filename and the handler

Notice that you may use any kind of variable type, as long as you specify it like I did
in the above example with <float>.

The second function returns the length of a bitmap, after reading it into the INPUT
vector. The number 2 represents the index of the bitmap in the previously defined file.