eigenfisherface.tex

automatic face recognition

\end{eqnarray}
Using a property of the orthonomal basis:
\begin{equation}
||\hat{\mathbf{x}}||_2 = ||\mathbf{W}^T \hat{\mathbf{x}}||_2,
\end{equation}
the Eq. \ref{eq:reconstructed} is
\begin{eqnarray}
i^* &=& \arg \min_i ||\mathbf{W}^T (\hat{\mathbf{x}} - \hat{\mathbf{x}}_i)||_2 \\
&=& \arg \min_i ||\mathbf{y} - \mathbf{y}_i||_2. 
\end{eqnarray}
Therefore, the recognition task using the reconstructed images is equivalent with the one using the PCA projected points without reconstruction. 

The result of occluded face shows another interesting capability of the PCA as a interpolation or a noise removal method. 



\begin{figure}[htbp]
\begin{center}
\begin{tabular}{ccc}
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=face01.eps,width=1.6cm}}
\end{minipage} &
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=face06.eps,width=1.6cm}}
\end{minipage} &
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=occludedface06.eps,width=1.6cm}}
\end{minipage}\\
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=re_face01.eps,width=1.6cm}}
\end{minipage} &
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=re_face06.eps,width=1.6cm}}
\end{minipage} &
\begin{minipage}[b]{0.2\linewidth}
  \centering
  \centerline{\epsfig{figure=re_occludedface06.eps,width=1.6cm}}
\end{minipage}
\end{tabular}
\end{center}
\caption{Top row: the orginal images. Bottom row: the reconstructed images.}
\label{fig:newset}
\end{figure}


\section{Fisherface}\label{S:Fisherface}

% The previous algorithm takes advantage of the fact that, under admittedly idealized conditions, the variation within class lies in a linear subspace of the image space. Hence, the classes are convex, and, therefore, linearly separable. One can perform dimensionality reduction using linear projection and still preserve linear separability. This is a strong argument in favor of using linear methods for dimensionality reduction in the face recognition problem, at least when one seeks insensitivity to lighting conditions.

  Since the learning set is labeled, it makes sense to use this information to build a more reliable method for reducing the dimensionality of the feature space. 
% Here we argue that using class specific linear methods for dimensionality reduction and simple classifiers in the reduced feature space, one may get better recognition rates than with either the Linear Subspace method or the Eigenface method. 
% Fisher's Linear Discriminant (FLD) \cite{fisher1936the} is an example of a {\it class specific method}, in the sense that it tries to ``shape'' the scatter in order to make it more reliable for classification. 
{\it Fisherface} method \cite{belhumeur1996eigenfaces} uses a class specific linear method, Fisher's Linear Discriminant (FLD) \cite{fisher1936the}, for dimensionality reduction and simple classifiers in the reduced feature space. 
This method selects $W$ in \cite{chellappa1995human} in such a way that the ratio of the between-class scatter and the within class scatter is maximized.

Again, 
let us consider a set of $N$ sample images $\{\mathbf{x}_{1},\ \mathbf{x}_{2},\ \ldots,\ \mathbf{x}_{N}\}$ taking values in an n-dimensional image space, and assume that each image belongs to one of $c$ classes $\{X_{1},X_{2},\ \ldots,X_{c}\}$.
  Let the between-class scatter matrix be defined as
\begin{equation}
S_{B}=\sum_{i=1}^{c}N_{i}(\mathbf{\mu}_{i}-\mathbf{\mu})(\mathbf{\mu}_{i}-\mathbf{\mu})^{T}
\end{equation}
and the within-class scatter matrix be defined as
\begin{equation}
S_{W}=\sum_{i=1}^{c}\sum_{\mathbf{x}_{k}\in X_{i}}(\mathbf{x}_{k}-\mathbf{\mu}_{i})(\mathbf{x}_{k}-\mathbf{\mu}_{i})^{T}
\end{equation}
where $\mathbf{\mu}_{i}$ is the mean image of class $X_{i}$, $N_{i}$ is the number of samples in class $X_{i}$, and $\mathbf{\mu}$ is the mean image of all samples. If $S_{W}$ is nonsingular, the optimal projection $W_{\text{opt}}$ is chosen as the matrix with orthonormal columns which maximizes the ratio of the determinant of the between-class scatter matrix of the projected samples to the determinant of the within-class scatter matrix of the projected samples, i.e.,
\begin{eqnarray}
W_{\text{opt}}&=&\arg\max_{W}\frac{|W^{T}S_{B}W|}{|W^{T}S_{W}W|} \nonumber \\
&=&[\mathbf{w}_{1}\mathbf{w}_{2}\text{ }...\text{ }\mathbf{w}_{m}] \label{eq:Fisherface4}
\end{eqnarray}
where $\{\mathbf{w}_i |i=1,2,\ \ldots,\ m\}$ is the set of generalized eigenvectors of $S_{B}$ and $S_{W}$ corresponding to the $m$ largest generalized eigenvalues $\{\lambda_{i}|i=1,2,\ \ldots,\ m\}$, i.e.,
\begin{equation}
S_{B}\mathbf{w}_{i}=\lambda_{i}S_{W}\mathbf{w}_{i},~~~~~~~i=1,2,\ldots,m.
\end{equation}
Note that there are at most $c-1$ nonzero generalized eigenvalues, and so an upper bound on $m$ is $c-1$, where $c$ is the number of classes. See \cite{duda1973pattern}.


  To illustrate the benefits of class specific linear projection, a low dimensional analogue to the classification problem in which the samples from each class lie near a linear subspace is shown. Fig.  \ref{fig:Fisherface2} is a comparison of PCA and FLD for a two-class problem in which the samples from each class are randomly perturbed in a direction perpendicular to a linear subspace. For this example, $N=20,\ n=2$, and $m=1$. So, the samples from each class lie near a line passing through the origin in the 2D feature space. Both PCA and FLD have been used to project the points from 2D down to 1D. Comparing the two projections in the figure, {\it PCA actually smears the classes together} so that they are no longer linearly separable in the projected space. It is clear that, although PCA achieves larger total scatter, FLD achieves greater between-class scatter, and, consequently, classification is simplified.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=74.51mm,height=75.48mm]{./Fisherface_images/image004.eps}
\end{center}
\caption{A comparison of principal component analysis (PCA) and Fisher's linear discriminant (FLD) for a two class problem where data for each class lies near a linear subspace.}
\label{fig:Fisherface2}
\end{figure}

  In the face recognition problem, one is confronted with the difficulty that the within-class scatter matrix $S_{W}\in \mathbb{R}^{n\times n}$ is always singular. This stems from the fact that the rank of $S_{W}$ is at most $N-c$, and, in general, the number of images in the learning set $N$ is much smaller than the number of pixels in each image $n$. This means that it is possible to choose the matrix $W$ such that the within-class scatter of the projected samples can be made exactly zero.

  In order to overcome the complication of a singular $S_{W}$, an alternative to the criterion in (\ref{eq:Fisherface4}) is proposed. This
method, named {\it Fisherfaces}, avoids this problem by projecting the image set to a lower dimensional space so that the resulting within-class scatter matrix $S_{W}$ is nonsingular. This is achieved by using PCA to reduce the dimension of the feature space to $N-c$, and then applying the standard FLD defined by (\ref{eq:Fisherface4}) to reduce the dimension to $c-1$. More formally, $W_{\text{opt}}$ is given by
\begin{equation}
W_{\text{opt}}^{T}=W_{\text{fld}}^{T}W_{\text{pca}}^{T}
\end{equation}
where
\begin{equation}
W_{\text{pca}}=\arg \max_{W}|W^{T}S_{T}W|
\end{equation}
\begin{equation}
W_{\text{fld}}\text{ }=\arg \max_{W}\frac{|W^{T}W_{\text{pca}}^{T}S_{B}W_{\text{pca}}W|}{|W^{T}W_{\text{pca}}^{T}S_{W}W_{\text{pca}}W|}
\end{equation}
Note that the optimization for $W_{\text{pca}}$ is performed over $n\times(N-c)$ matrices with orthonormal columns, while the optimization for $W_{\text{fld}}$ is performed over $(N-c)\times m$ matrices with orthonormal columns. In computing $W_{\text{pca}}$, we have thrown away only the smallest c-l principal components.

%   There are certainly other ways of reducing the within class scatter while preserving between-class scatter. For example, a second method which we are currently investigating chooses $W$ to maximize the between-class scatter of the projected samples after having first reduced the within-class scatter. Taken to an extreme, we can maximize the between-class scatter of the projected samples subject to the constraint that the within-class scatter is zero, i.e.,
% \begin{equation}
% W_{\text{opt}}=\arg \max_{W \in \mathcal{W}}|W^{T}S_{B}W|
% \end{equation}
% where $\mathcal{W}$ is the set of $n\times m$ matrices with orthonormal columns contained in the kernel of $S_{W}$.

\section{Experimental Results}\label{S:Comparison}

The PIE database is used in the experiments. The PIE database contains 21 face images of 68 people having a large illumination variations. A few sample images from PIE database are shown in Fig. \ref{fig:PIE}. The former 16 images of each person are used as a training set and the latter 5 images are used as a testing set. 


\begin{figure}[htbp]
\begin{tabular}{ccccc}
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/01/img01.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/02/img01.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/03/img01.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/04/img01.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/05/img01.eps,width=1.4cm}}
\end{minipage}\\
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/01/img02.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/02/img02.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/03/img02.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/04/img02.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/05/img02.eps,width=1.4cm}}
\end{minipage}\\
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/01/img03.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/02/img03.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/03/img03.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/04/img03.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/05/img03.eps,width=1.4cm}}
\end{minipage}\\
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/01/img04.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/02/img04.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/03/img04.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/04/img04.eps,width=1.4cm}}
\end{minipage} &
\begin{minipage}[b]{0.15\linewidth}
  \centering
  \centerline{\epsfig{figure=PIE/05/img04.eps,width=1.4cm}}
\end{minipage}
\end{tabular}
\caption{Examples from PIE database. Original image size 48x40.}
\label{fig:PIE}
\end{figure}

The Fig. \ref{fig:EigenFisher} shows a comparison of the Eigenface and Fisherface methods with respect to the recognition rate versus number of feature dimensions used. The Fig. \ref{fig:CMS} shows an evaluation based on cumulative match score (CMS) \cite{phillips1997the} using 40 dimensional feature vectors. 


\begin{figure}[htbp]
\begin{center}
  \includegraphics[width=9.0cm]{EigenFisher1.eps}
\end{center}
\caption{Face recognition experiment: Eigenface v.s. Fisherface}
\label{fig:EigenFisher}
\end{figure} 

\begin{figure}[htbp]
\begin{center}
  \includegraphics[width=9.0cm]{CMS1.eps}
\end{center}
\caption{Cumulative Match Score: Eigenface v.s. Fisherface. 40 dimensional features are used.}
\label{fig:CMS}
\end{figure} 

\section{Conclusion}\label{S:Conclusion}

The Eigenface and Fisherface method were investigated and compared. 
The comparative experiment showed that the Fisherface method outperformed the Eigenface method. 
The usefulness of the Fisherface method under varying illumination was verified. 

% To start a new column (but not a new page) and help balance the last-page
% column length use \vfill\pagebreak.
% -------------------------------------------------------------------------
% \vfill
% \pagebreak

% References should be produced using the bibtex program from suitable
% BiBTeX files (here: strings, refs, manuals). The IEEEbib.bst bibliography
% style file from IEEE produces unsorted bibliography list.
% -------------------------------------------------------------------------
%\nocite{*}
% \bibliographystyle{IEEEbib}
\bibliographystyle{amsplain}
\bibliography{EigenFisherFace}

\end{document}