adapt.tex

隐马尔科夫模型工具箱

%/* ----------------------------------------------------------- */
%/*                                                             */
%/*                          ___                                */
%/*                       |_| | |_/   SPEECH                    */
%/*                       | | | | \   RECOGNITION               */
%/*                       =========   SOFTWARE                  */ 
%/*                                                             */
%/*                                                             */
%/* ----------------------------------------------------------- */
%/*         Copyright: Microsoft Corporation                    */
%/*          1995-2000 Redmond, Washington USA                  */
%/*                    http://www.microsoft.com                 */
%/*                                                             */
%/*   Use of this software is governed by a License Agreement   */
%/*    ** See the file License for the Conditions of Use  **    */
%/*    **     This banner notice must not be removed      **    */
%/*                                                             */
%/* ----------------------------------------------------------- */
\mychap{HMM Adaptation}{Adapt}

\sidepic{headapt}{80}{
Chapter~\ref{c:Training} described how the parameters are estimated
for plain continuous density HMMs within \HTK, primarily using the
embedded training tool \htool{HERest}. Using the training strategy
depicted in figure~\ref{f:subword}, together with other techniques can
produce high performance speaker independent acoustic models for a 
large vocabulary recognition system. However it is possible to build
improved acoustic models by tailoring a model set to a specific
speaker. By collecting data from a speaker and training
a model set on this speaker's data alone, the speaker's characteristics
can be modelled more accurately. Such systems are commonly 
known as \textit{speaker dependent} systems, and on a typical word
recognition task, may have half the errors of a speaker
independent system. The drawback of speaker dependent systems is that
a large amount of data (typically hours) must be collected in order to
obtain sufficient model accuracy.
}

Rather than training speaker dependent models, 
\textit{adaptation} techniques can be applied. In this case, by using
only a small amount of data from a new speaker, a good speaker
independent system model set can be adapted to better fit the
characteristics of this new speaker.

Speaker adaptation techniques can be used in various different
modes\index{adaptation!adaptation modes}. If
the true transcription of the adaptation data is known then
it is termed 
\textit{supervised adaptation}\index{adaptation!supervised adaptation}, 
whereas if the adaptation
data is unlabelled then it is termed 
\textit{unsupervised adaptation}\index{adaptation!unsupervised
adaptation}.
In the case where all the adaptation data is available in one block,
e.g. from a speaker enrollment session, then this termed \textit{static
adaptation}. Alternatively adaptation can proceed incrementally as
adaptation data becomes available, and this is termed 
\textit{incremental adaptation}.  
% \htool{HVite} can provide unsupervised incremental adaptation.

\HTK\ provides two tools to adapt continuous density HMMs. 
\htool{HEAdapt}\index{headapt@\htool{HEAdapt}} 
performs offline supervised adaptation
using maximum likelihood linear regression (MLLR) and/or 
maximum a-posteriori (MAP) adaptation, while  
unsupervised adaptation is supported by \htool{HVite} (using only MLLR).
In this case \htool{HVite} not only performs recognition, but
simultaneously adapts the model set as the data becomes available
through recognition. Currently, MLLR adaptation can be applied in both
incremental and static modes while MAP supports only static
adaptation. If MLLR and MAP adaptation is to be performed
simultaneously using 
\htool{HEAdapt} in the same pass, 
then the restriction is that the entire
adaptation must be performed statically\footnote{
By using two passes,
one could perform incremental MLLR in the first pass (saving the new
model or transform), followed by a second pass, this time using MAP 
adaptation.}.

This chapter describes the supervised adaptation tool \htool{HEAdapt}.
The first sections of the chapter
give an overview of MLLR and MAP adaptation and 
this is followed by a section describing the general
usages of \htool{HEAdapt} to build simple and more complex adapted
systems. The chapter concludes with a section detailing the various
formulae used by the adaptation tool.
The use of \htool{HVite} to perform unsupervised adaptation is 
discussed in section~\ref{s:unsup_adapt}. 

\mysect{Model Adaptation using MLLR}{mllr}

\mysubsect{Maximum Likelihood Linear Regression}{whatismllr}

Maximum likelihood linear regression or MLLR\index{adaptation!MLLR}
computes a set of transformations that will reduce the mismatch
between an initial model set and the adaptation data\footnote{
MLLR can also be used to perform environmental compensation by
reducing the mismatch due to channel or additive noise effects.}.
More specifically MLLR is a model adaptation technique
that estimates a set of linear transformations for the mean and
variance parameters of a Gaussian mixture HMM system. 
%The set of
%transformations are estimated so to as to maximise the likelihood of the
%adaptation data. 
The effect of these transformations is to shift the
component means and alter the variances in the initial system 
so that each state in the HMM system is more likely to generate the 
adaptation data.
Note that due to computational reasons, MLLR is only implemented
within \HTK\ for diagonal covariance, single stream, continuous density
HMMs.

The transformation matrix used to give a new estimate of the adapted mean is
given by
\hequation{ 
        \hat{\bm{\mu}} = \bm{W}\bm{\xi}, 
}{mtrans}
where $\bm{W}$ is the $n \times \left( n + 1 \right)$
transformation matrix (where $n$ is the dimensionality of the data)
and $\bm{\xi}$ is the extended mean vector,
\[
        \bm{\xi} = \left[\mbox{ }w\mbox{ }\mu_1\mbox{ }\mu_2\mbox{ }\dots\mbox{ }\mu_n\mbox{ }\right]^T
\]
where $w$ represents a bias offset whose value is fixed (within \HTK) at 1.\\
Hence $\bm{W}$ can be decomposed into
\hequation{
        \bm{W} = \left[\mbox{ }\bm{b}\mbox{ }\bm{A}\mbox{ }\right]
}{decompmtrans}
where $\bm{A}$ represents an $n \times n$
transformation matrix and $\bm{b}$ represents a bias vector.

The transformation matrix $\bm{W}$ is obtained by solving a
maximisation problem using the \textit{Expectation-Maximisation}
(EM) technique. This technique is also used to compute the variance
transformation matrix. Using EM results in the maximisation of a
standard \textit{auxiliary function}. (Full details are available in
section~\ref{s:mllrformulae}.)

\mysubsect{MLLR and Regression Classes}{reg_classes}
\index{adaptation!regression tree}
This adaptation method can be applied in a very flexible manner,
depending on the amount of adaptation data that is available. If a
small amount of data is available then a \textit{global} adaptation transform 
\index{adaptation!global transforms} can be generated. A global transform 
(as its name suggests) is applied
to every Gaussian component in the model set. However as more
adaptation data becomes available, improved adaptation is possible by
increasing the number of transformations. Each transformation is now
more specific and applied to certain groupings of Gaussian components.
For instance the Gaussian components could be grouped into the broad 
phone classes: silence, vowels, stops, glides, nasals, fricatives, etc.
The adaptation data could now be used to construct more specific broad
class transforms to apply to these groupings.

Rather than specifying static component groupings or classes, a robust
and dynamic method is used for the construction of further transformations
as more adaptation data becomes available. MLLR makes
use of a \textit{regression class tree} to group the Gaussians in the
model set, so that the set of transformations to be estimated can be
chosen according to the amount and type of adaptation data that is
available. The tying of each transformation across a number of mixture
components makes it possible to adapt distributions for which there
were no observations at all. With this process all models can be
adapted and the adaptation process is dynamically refined when more
adaptation data becomes available.\\

The regression class tree 
is constructed so as to cluster
together components that are close in acoustic
space, so that similar components can be transformed in a similar way.
Note that the tree is built using the original speaker independent
model set, and is thus independent of any new speaker.
The tree is constructed with a centroid splitting algorithm, which uses 
a Euclidean distance measure. For more details see
section~\ref{s:hhedregtree}.
The terminal nodes or leaves of the tree specify the final component
groupings, and are termed the \textit{base
(regression) classes}. Each Gaussian component of a model set belongs 
to one particular base class. The tool \htool{HHEd} can
be used to build a binary regression class tree, and to label each
component with a base class number.  Both the tree and component base
class numbers are saved automatically as part of the MMF. Please refer to
section~\ref{s:regtreemods} and section~\ref{s:hhedregtree} for
further details.

\sidefig{regtree1}{55}{A binary regression tree}{4}{}
Figure~\ref{f:regtree1} shows a simple example of a binary regression
tree with four base classes, denoted as $\{C_4, C_5, C_6,
C_7\}$. During ``dynamic'' adaptation, the
occupation counts are accumulated for each of the regression base
classes. The diagram shows a solid arrow and circle
(or node), indicating that there is sufficient data for a transformation
matrix to be generated using the data associated with that class. A
dotted line and circle indicates that there is insufficient
data. For example neither node 6 or 7 has sufficient data; however
when pooled at node 3, there is sufficient adaptation data.
 The amount of data that is ``determined'' as sufficient is set
by the user as a command-line option to \htool{HEAdapt} (see reference
section~\ref{s:HEAdapt}).

\htool{HEAdapt} uses a top-down approach to traverse the regression
class tree. Here the search starts at the root node and progresses
down the tree generating transforms only for those nodes which
\begin{enumerate}
\item have sufficient data \textbf{and}
\item are either terminal nodes (i.e. base classes) \textbf{or} have
any children without sufficient data.
\end{enumerate}

In the example shown in figure~\ref{f:regtree1}, transforms are constructed
only for regression nodes 2, 3 and 4, which can be denoted as
${\bf W}_2$, ${\bf W}_3$ and ${\bf W}_4$. Hence when the transformed
model set is required, the transformation matrices (mean and variance)
are applied in the following fashion to the Gaussian components in
each base class:-
\[
        \left\{
        \begin{array}{ccl}
                {\bf W}_2 & \rightarrow & \left\{C_5\right\} \\
                {\bf W}_3 & \rightarrow & \left\{C_6, C_7\right\} \\
                {\bf W}_4 & \rightarrow & \left\{C_4\right\}
        \end{array}
        \right\}
\]

At this point it is interesting to note that the global adaptation
case is the same as a tree with just a root node, and is in fact
treated as such.

\mysubsect{Transform Model File Format}{tmfs}

\htool{HEAdapt} estimates the required transformation statistics and can
either output a transformed MMF or a transform model file 
(TMF)\index{adaptation!transform model file}. The
advantage in storing the transforms as opposed to an adapted