hw97.tex

Mitchell的《机器学习〉随书源码

Another fun thing to do, if you didn't already try it in the last
question of the assignment, is to use the image package
to view the weights on connections in graphical form; you will find
routines for creating and writing images, if you want to play around
with visualizing your network weights.

Finally, the point of this assignment is for you to obtain first-hand
experience in working with neural networks; it is {\bf not} intended as an
exercise in C hacking.  An effort has been made to keep the image package
and neural network package as simple as possible.  If you need
clarifications about how the routines work, don't hesitate to ask.

\subsection{The neural network package}

As mentioned earlier, this package implements three-layer fully-connected
feedforward neural networks, using a backpropagation weight tuning
method.  We begin with a brief description of the data structure,
a {\tt BPNN} ({\tt B}ack{\tt P}rop{\tt N}eural{\tt N}et).

All unit values and weight values are stored as {\tt double}s in a
{\tt BPNN}.

Given a {\tt BPNN *net}, you can get the number of input, hidden,
and output units with {\tt net->input\_n}, {\tt net->hidden\_n},
and {\tt net->output\_n}, respectively.

Units are all indexed from $1$ to $n$,
where $n$ is the number of units in the layer.  To get the value
of the {\tt k}th unit in the input, hidden, or output layer, use
{\tt net->input\_units[k]}, {\tt net->hidden\_units[k]}, or
{\tt net->output\_units[k]}, respectively.

The target vector is assumed to have the same number of values as the number
of units in the output layer, and it can be accessed via {\tt net->target}.
The {\tt k}th target value can be accessed by {\tt net->target[k]}.

To get the value of the weight connecting the {\tt i}th input unit
to the {\tt j}th hidden unit, use {\tt net->input\_weights[i][j]}.
To get the value of the weight connecting the {\tt j}th hidden unit
to the {\tt k}th output unit, use {\tt net->hidden\_weights[j][k]}.

The routines are as follows:

\begin{description}
\item {\tt void bpnn\_initialize(seed)\newline
int seed;}

This routine initializes the neural network package.  It should be
called before any other routines in the package are used.  Currently,
its sole purpose in life is to initialize the random number generator
with the input {\tt seed}.

\item {\tt BPNN *bpnn\_create(n\_in, n\_hidden, n\_out)\newline
int n\_in, n\_hidden, n\_out;}

Creates a new network with {\tt n\_in} input units, {\tt n\_hidden} hidden
units, and {\tt n\_output} output units.  All weights in the network
are randomly initialized to values in the range $[-1.0, 1.0]$.  Returns
a pointer to the network structure.  Returns {\tt NULL} if the routine
fails.

\item {\tt void bpnn\_free(net)\newline
BPNN *net;}

Takes a pointer to a network, and frees all memory associated with
the network.

\item {\tt void bpnn\_train(net, learning\_rate, momentum, erro, errh)\newline
BPNN *net;\newline double learning\_rate, momentum;\newline
double *erro, *errh;}

Given a pointer to a network, runs one pass of the backpropagation algorithm.
Assumes that the input units and target layer have been properly set up.
{\tt learning\_rate} and {\tt momentum} are assumed to be values between
$0.0$ and $1.0$.  {\tt erro} and {\tt errh} are pointers to doubles, which
are set to the sum of the $\delta$ error values on the output units
and hidden units, respectively.

\item {\tt void bpnn\_feedforward(net)\newline
BPNN *net;}

Given a pointer to a network, runs the network on its current input
values.

\item {\tt BPNN *bpnn\_read(filename)\newline
char *filename;}

Given a filename, allocates space for a network, initializes it with the
weights stored in the network file, and returns a pointer to this new
{\tt BPNN}.  Returns {\tt NULL} on failure.

\item {\tt void bpnn\_save(net, filename)\newline
BPNN *net;\newline char *filename;}

Given a pointer to a network and a filename, saves the network to that
file.

\end{description}

\subsection{The image package}

The image package provides a set of routines for manipulating PGM images.
An image is a rectangular grid of pixels; each pixel has an integer value
ranging from 0 to 255.  Images are indexed by rows and columns; row 0
is the top row of the image, column 0 is the left column of the image.

\begin{description}

\item {\tt IMAGE *img\_open(filename)\newline char *filename;}

Opens the image given by {\tt filename}, loads it into a new {\tt IMAGE}
data structure, and returns a pointer to this new structure.
Returns {\tt NULL} on failure.

\item {\tt IMAGE *img\_creat(filename, nrows, ncols)\newline
char *filename;\newline
int nrows, ncols;}

Creates an image in memory, with the given filename, of dimensions
{\tt nrows} $\times$ {\tt ncols}, and returns a pointer to this image.
All pixels are initialized to 0.  Returns {\tt NULL} on failure.

\item {\tt int ROWS(img)\newline IMAGE *img;}

Given a pointer to an image, returns the number of rows the image has.

\item {\tt int COLS(img)\newline IMAGE *img;}

Given a pointer to an image, returns the number of columns the image has.

\item {\tt char *NAME(img)\newline IMAGE *img;}

Given a pointer to an image, returns a pointer to its base filename
(i.e., if the full
filename is {\tt /usr/joe/stuff/foo.pgm}, a pointer to the string
{\tt foo.pgm} will be returned).

\item {\tt int img\_getpixel(img, row, col)\newline IMAGE *img;\newline
int row, col;}

Given a pointer to an image and row/column coordinates, this routine returns
the value of the pixel at those coordinates in the image.

\item {\tt void img\_setpixel(img, row, col, value)\newline
IMAGE *img;\newline int row, col, value;}

Given a pointer to an image and row/column coordinates, and an integer
{\tt value}
assumed to be in the range $[0, 255]$, this routine sets the pixel
at those coordinates in the image to the given value.

\item {\tt int img\_write(img, filename)\newline
IMAGE *img;\newline char *filename;}

Given a pointer to an image and a filename, writes the image to disk with
the given filename.  Returns 1 on success, 0 on failure.

\item {\tt void img\_free(img)\newline IMAGE *img;}

Given a pointer to an image, deallocates all of its associated memory.

\item {\tt IMAGELIST *imgl\_alloc()}

Returns a pointer to a new {\tt IMAGELIST} structure, which is really just
an array of pointers to images.  Given an {\tt IMAGELIST *il},
{\tt il->n} is the number of images in the list.  {\tt il->list[k]}
is the pointer to the {\tt k}th image in the list.

\item {\tt void imgl\_add(il, img)\newline
IMAGELIST *il;\newline IMAGE *img;}

Given a pointer to an imagelist and a pointer to an image, adds the image
at the end of the imagelist.

\item {\tt void imgl\_free(il)\newline IMAGELIST *il;}

Given a pointer to an imagelist, frees it.  Note that this does not
free any images to which the list points.

\item {\tt void imgl\_load\_images\_from\_textfile(il, filename)\newline
IMAGELIST *il;\newline char *filename;}

Takes a pointer to an imagelist and a filename.  {\tt filename} is
assumed to specify a file which is a list of pathnames of images,
one to a line.  Each image file in this list is loaded into memory
and added to the imagelist {\tt il}.

\end{description}

\subsection {\tt hidtopgm}

{\tt hidtopgm} takes the following fixed set of arguments:

{\tt hidtopgm} {\it net-file image-file x y n}

\begin{description}

\item {\it net-file} is the file containing the network in which
the hidden unit weights are to be found.

\item {\it image-file} is the file to which the derived image will
be output.

\item {\it x} and {\it y} are the dimensions in pixels of the image
on which the network was trained.

\item {\it n} is the number of the target hidden unit. {\it n} may
range from 1 to the total number of hidden units in the network.

\end{description}

\subsection {\tt outtopgm}

{\tt outtopgm} takes the following fixed set of arguments:

{\tt outtopgm} {\it net-file image-file x y n}

\begin{description}
\item This is the same as hidtopgm, for output units instead of input units.
Be sure you specify x to be 1 plus the number of hidden units, so that you get
to see the weight $w_{0}$ as well as weights associated with the hidden units.
For example, to see the weights for output number 2 of a network containing 3
hidden units, do this:

outtopgm pose.net pose-out2.pgm 4 1 2

\item {\it net-file} is the file containing the network in which
the hidden unit weights are to be found.

\item {\it image-file} is the file to which the derived image will
be output.

\item {\it x} and {\it y} are the dimensions of the hidden
units, where {\it x} is always 1 + the number of hidden units
specified
for the network, and {\it y} is always 1.

\item {\it n} is the number of the target output unit. {\it n} may
range from 1 to the total number of output units for the network.

\end{description}

\end{document}