hw97.tex

Mitchell的《机器学习〉随书源码

\documentstyle[12pt]{cmu-art}
\def\BOX#1{\fbox{\tt #1}}
\begin{document}

\title{Homework 3: Neural networks and face images }
\author{15-681: Machine Learning \\
Tom Mitchell \\
Carnegie Mellon University}
\date{Due: October 16, 1997}
\maketitle

\section{Introduction}

This assignment gives you an opportunity to apply neural network learning to
the problem of face recognition.  It is broken into two parts.  For the first
part, which you must do alone, you will experiment with a neural network
program to train a sunglasses recognizer, a face recognizer, and a pose
recognizer.  For the second part, which is optional and for extra credit only,
you have the option of working with a group of 1 or 2 other students to study
some issue of your own choosing.  The face images you will use are faces of
students from earlier Machine Learning classes.

You will not need to do significant amounts of coding for this assignment,
and you should not let the size of this document scare you, but training
your networks will take time.  It is recommended that you read the
assignment in its entirety first, and start early.

\subsection{The face images}

The image data can be found in {\tt /afs/cs/project/theo-8/faceimages/faces}.
This directory contains 20 subdirectories, one for each person, named by
userid.  Each of these directories contains several different face images of
the same person.

You will be interested in the images with the following naming convention:

{\tt <userid>\_<pose>\_<expression>\_<eyes>\_<scale>.pgm}

\begin{itemize}

\item {\tt <userid>} is the user id of the person in the image, and this
field has 20 values: an2i, at33, boland, bpm, ch4f, cheyer, choon, danieln,
glickman, karyadi, kawamura, kk49, megak, mitchell, night, phoebe, saavik,
steffi, sz24, and tammo.

\item {\tt <pose>} is the head position of the person, and this field
has 4 values: straight, left, right, up.

\item {\tt <expression>} is the facial expression of the person, and this
field has 4 values: neutral, happy, sad, angry.

\item {\tt <eyes>} is the eye state of the person, and this field has
2 values: open, sunglasses.

\item {\tt <scale>} is the scale of the image, and this field has 3
values: 1, 2, and 4.  1 indicates a full-resolution image ($128$ columns
$\times$ $120$ rows); 2 indicates a half-resolution image ($64 \times 60$);
4 indicates a quarter-resolution image ($32 \times 30$).  For this
assignment, you will be using the quarter-resolution images for experiments,
to keep training time to a manageable level.

\end{itemize}

If you've been looking closely in the image directories, you may notice
that some images have a {\tt .bad} suffix rather than the {\tt .pgm}
suffix.  As it turns out, 16 of the 640 images taken have glitches due to
problems with the camera setup; these are the {\tt .bad} images.
Some people had more glitches than others, but everyone who got ``faced''
should have at least 28 good face images (out of the 32 variations
possible, discounting scale).

\subsection{Viewing the face images}

To view the images, you can use the program {\tt xv}.  This is available
as {\tt /usr/local/bin/xv} on Andrew machines, and
{\tt /usr/misc/.X11-others/bin/xv} or {\tt /usr/local/bin/xv}
on CS machines.  {\tt xv} handles
a variety of image formats, including the PGM format in which our
face images are stored.  While we won't go into detail about {\tt xv}
in this document, we will quickly describe the basics you need to know
to use {\tt xv}.

To start {\tt xv}, just specify one or more images on the command line,
like this:

{\tt xv /afs/cs/project/theo-8/faceimages/faces/glickman/glickman\_straight\_happy\_open\_4.pgm}

This will bring up an X window displaying the face.  Clicking the right button
in the image window will toggle a control panel with a variety of buttons.
The {\tt Dbl Size} button doubles the displayed size of the image every time
you click on it.  This will be useful for viewing the quarter-resolution
images, as you might imagine.

You can also obtain pixel values by holding down the left button while moving
the pointer in the image window.  A text bar will be displayed, showing you
the image coordinates and brightness value where the pointer is located.

To quit {\tt xv}, just click on the {\tt Quit} button or type {\tt q}
in one of the {\tt xv} windows.

\subsection{The neural network and image access code}

We're supplying C code for a three-layer fully-connected feedforward
neural network which uses the backpropagation algorithm to tune its weights.
To make life as easy as possible, we're also supplying you with an image
package for accessing the face images, as well as the top-level
program for training and testing, as a skeleton for you to modify.
To help explore what the nets actually learn, you'll also find a
utility program for visualizing hidden-unit weights as images.

The code is located in {\tt /afs/cs/project/theo-8/faceimages/code}.  Copy all
of the files in this area to your own directory, and type {\tt make}. 
Note: take care to use {\tt cp *} instead of {\tt cp *.*} in order 
to ensure that you get the {\tt Makefile}. When
the compilation is done, you should have one executable program: {\tt
facetrain}.  Briefly, {\tt facetrain} takes lists of image files as input, and
uses these as training and test sets for a neural network.  {\tt facetrain}
can be used for training and/or recognition, and it also has the capability to
save networks to files.

The code has been compiled and tested successfully on CS-side Alphas,
DecStations, Sun SPARC-2s, and IBM PowerPC's, and Andrew-side DecStations and
Sun SPARC-5s.  If you wish to use the code on some other platform, feel free,
but be aware that the code has only been tested on these platforms.  

Details of the routines, explanations of the source files, and related
information can be found in Section \ref{docs} of this handout.

\section{The Assignment}

\subsection{Part I (required)}

Turn in a short write-up of your answers to the questions found in the
following sequence of initial experiments.

\begin{enumerate}

\item Issue the following command in your home directory to obtain
the training and test set data for this assignment:

{\tt cp /afs/cs/project/theo-8/faceimages/trainset/*.list .}

\item The code you have been given is currently set up to learn to recognize
the person with userid {\tt glickman}.  Modify this code to implement a
``sunglasses'' recognizer; i.e., train a neural net which, when given an image
as input, indicates whether the face in the image is wearing sunglasses, or
not. Refer to the beginning of Section 3 for an overview of how to
make changes to this code.

\item Train a network using the default learning parameter settings (learning
rate 0.3, momentum 0.3) for 75 epochs, with the following command:

{\tt facetrain -n shades.net -t straightrnd\_train.list -1 straightrnd\_test1.list}
\newline {\tt -2 straightrnd\_test2.list -e 75}

{\tt facetrain}'s arguments are described in Section \ref{RUNFACE},
but a short description is in order here.  {\tt shades.net} is the name of
the network file which will be saved when training is finished.
{\tt straightrnd\_train.list}, {\tt straightrnd\_test1.list}, and
{\tt straightrnd\_test2.list} are text files which specify the training
set (70 examples) and two test sets (34 and 52 examples), respectively.

This command creates and trains your net on a randomly chosen sample of 70 of
the 156 ``straight'' images, and tests it on the remaining 34 and 52 randomly
chosen images, respectively.  One way to think of this test strategy is that
roughly $\frac{1}{3}$ of the images ({\tt straightrnd\_test2.list}) have been
held over for testing.  The remaining $\frac{2}{3}$ have been used for a train
and cross-validate strategy, in which $\frac{2}{3}$ of these are being used
for as a training set ({\tt straightrnd\_train.list}) and $\frac{1}{3}$ are
being used for the validation set to decide when to halt training ({\tt
straightrnd\_test1.list}).

\item What code did you modify?  What was the maximum classification accuracy 
achieved on the training set?  How many epochs did it take to reach this
level?  How about for the validation set?  The test set?  Note that if you run
it again on the same system with the same parameters and input, you should get
exactly the same results because, by default, the code uses the same seed to
the random number generator each time. You will need to read Section
3.1.2 carefully in order to be able to interpret your experiments and
answer these questions.

\item Now, implement a 1--of--20 face recognizer; i.e. implement a neural 
net that accepts an image as input, and outputs the userid of the person.  To
do this, you will need to implement a different output encoding (since you
must now be able to distinguish among 20 people).  (Hint: leave learning rate
and momentum at 0.3, and use 20 hidden units).

\item As before, train the network, this time for 100 epochs:

{\tt facetrain -n face.net -t straighteven\_train.list -1 straighteven\_test1.list}
\newline {\tt -2 straighteven\_test2.list -e 100}

You might be wondering why you are only training on samples from a limited
distribution (the ``straight'' images).  The essential reason is training
time.  If you have access to a very fast machine (anything slower than an
Alpha or Sun4 may be too slow), then you are welcome to do these experiments
on the entire set (replace {\tt straight} with {\tt all} in the command above.
Otherwise, stick to the ``straight'' images.

The difference between the {\tt straightrnd\_*.list} and the {\tt
straighteven\_*.list} sets is that while the former divides the images purely
randomly among the training and test sets, the latter ensures a relatively
even distribution of each individual's images over the sets.  Because we have
only 7 or 8 ``straight'' images per individual, failure to distribute them
evenly would result in testing our network the most on those faces on which it
was trained the least.

\item Which parts of the code was it necessary to modify this time?
How did you encode the outputs?  What was the maximum classification accuracy
achieved on the training set?  How many epochs did it take to reach this
level?  How about for the validation and test set?

\item Now let's take a closer look at which images the net may have failed
to classify:

{\tt facetrain -n face.net -T -1 straighteven\_test1.list -2 straighteven\_test2.list}

Do there seem to be any particular commonalities between the misclassified
images?

\item Implement a pose recognizer; i.e. implement a neural net which,
when given an image as input, indicates whether the person in the image
is looking straight ahead, up, to the left, or to the right.  You
will also need to implement a different output encoding for this task.
(Hint: leave learning rate and momentum at
0.3, and use 6 hidden units).

\item Train the network for 100 epochs, this time on samples drawn
from all of the images:

{\tt facetrain -n pose.net -t all\_train.list -1 all\_test1.list}
\newline {\tt -2 all\_test2.list -e 100}

Since the pose-recognizing network should have substantially fewer weights to
update than the face-recognizing network, even those of you with slow machines
can get in on the fun of using all of the images.  In this case, 260 examples
are in the training set, 140 examples are in test1, and 193 are in test2.

\item How did you encode your outputs this time?
What was the maximum classification accuracy achieved on the training set?
How many epochs did it take to reach this level?  How about for each test set?

\item Now, try taking a look at how backpropagation tuned the weights
of the hidden units with respect to each pixel.  First type {\tt make
hidtopgm} to compile the utility on your system.  Then, to visualize the