adapt.tex

隐马尔科夫模型工具箱

MMF is that the TMFs are considerably smaller than MMFs (especially
triphone MMFs). This
section describes the format of the transform model file in detail.


The mean transformation matrix is stored as a block diagonal
transformation matrix.
The example block diagonal matrix ${\bf A}$ shown below contains three
blocks. The first block represents the transformation for only the
static components of the feature vector, while the second represents
the deltas and the third the accelerations. This block diagonal matrix
example makes the assumption that for the transformation, there is no
correlation between the statics, deltas and delta deltas. In practice
this assumption works quite well.
\[
        {\bf A} \; = \; \left(
                        \begin{array}{ccc}
                        {\bf A}_s & {\bf 0}        & {\bf 0}       \\
                        {\bf 0}   & {\bf A}_\Delta & {\bf 0}       \\
                        {\bf 0}   & {\bf 0}        & {\bf A}_{\Delta^2}
                        \end{array}
                        \right)
\]

This format reduces the number of transformation parameters required
to be learnt, making the adaptation process faster. It also reduces
the adaptation data required per transform when compared with the full
case. When comparing the storage requirements, the 3 block diagonal
matrix requires much less storage capacity than the full transform matrix.
Note that for convenience a full transformation matrix is also stored 
as a block diagonal matrix, only in this case there is a single block.

The variance transformation is a diagonal matrix and as such is simply
stored as a vector.

\noindent
Figure~\ref{f:exampletmf} shows a simple example of a TMF. In this
case the feature vector has nine dimensions, and the mean transform has
three diagonal blocks. 
The TMF can be saved in ASCII or binary format. The user header is
always output in ascii.
The first two fields are speaker descriptor fields. The next field
\texttt{<MMFID>}, the MMF identifier, is obtained from the global
options macro in the MMF, while the regression class tree identifier
\texttt{<RCID>} is obtained from the regression tree macro name in the
MMF.
If global adaptation is being performed, then the \texttt{<RCID>} will
contain the identifier \texttt{global}, since a tree is unnecessary in
the global case.
Note that the MMF and regression class tree identifiers are set within the
MMF using the tool \htool{HHEd}. 
The final two fields are optional, but \htool{HEAdapt} outputs these
anyway for the user's convenience. These can be edited at any time (as
can all the fields if desired, but editing \texttt{<MMFID>} and
\texttt{<RCID>} fields should be avoided). The \texttt{<CHAN>} field
should represent the adaptation data recording environment. Examples
could be a particular microphone name, telephone channel or various
background noise conditions. The \texttt{<DESC>} allow the user to
enter any other information deemed useful. An example could be the
speaker's dialect region.

\sideprog{exampletmf}{70}{A Simple example of a TMF}{
\hmkw{UID} djk \\
\hmkw{NAME} Dan Kershaw \\
\hmkw{MMFID} ECRL\_UK\_XWRD \\
\hmkw{RCID} global \\
\hmkw{CHAN} Standard \\
\hmkw{DESC} None \\
\hmkw{NBLOCKS} 3 \\
\hmkw{NODETHRESH} 700.0 \\
\hmkw{NODEOCC} 1 24881.8 \\
\hmkw{TRANSFORM} 1 \\
\> \hmkw{MEAN\_TR} 3 \\
\>\>\hmkw{BLOCK} 1 \\
\>\>\> \mbox{ }0.942 -0.032 -0.001 \\
\>\>\> -0.102  \mbox{ }0.922 -0.015 \\
\>\>\> -0.016  \mbox{ }0.045  \mbox{ }0.910 \\
\>\>\hmkw{BLOCK} 2 \\                          
\>\>\> \mbox{ }1.021 -0.032 -0.011 \\
\>\>\> -0.017  \mbox{ }1.074 -0.043 \\
\>\>\> -0.099  \mbox{ }0.091  \mbox{ }1.050 \\
\>\>\hmkw{BLOCK} 3 \\            
\>\>\>  \mbox{ }1.028  \mbox{ }0.032  \mbox{ }0.001 \\
\>\>\> -0.012  \mbox{ }1.014 -0.011 \\
\>\>\> -0.091 -0.043  \mbox{ }1.041 \\
\>\hmkw{BIASOFFSET} 9 \\
\>\> -0.357 \mbox{ }0.001 -0.002 \mbox{ }0.132 \mbox{ }0.072 \\
\>\>\mbox{ }0.006 \mbox{ }0.150 \mbox{ }0.138 \mbox{ }0.198 \\
\>\hmkw{VARIANCE\_TR} 9  \\
\>\> \mbox{ }0.936 \mbox{ }0.865 \mbox{ }0.848 \mbox{ }0.832 \mbox{ }0.829 \\
\>\>\mbox{ }0.786 \mbox{ }0.947 \mbox{ }0.869 \mbox{ }0.912
}{}
Whenever a TMF is being used (in conjunction with an MMF), the MMF
identifier in the MMF is checked against that in the TMF. These
\textbf{must} match since the TMF is dependent on the model set it was
constructed from. Also unless the \texttt{<RCID>} field is set to
\texttt{global}, it is also checked for consistency against the
regression tree identifier in the MMF.

The rest of the TMF contains a further information header, followed by
all the transforms. The information header contains necessary
transform set information such as the number of blocks used, node occupation
threshold used, and the node occupation counts. Each
transform has a regression class identifier number, the mean
transformation matrix ${\bf A}$, an optional bias vector ${\bf b}$ (as
in equation~\ref{e:decompmtrans}) and
an optional variance transformation diagonal matrix ${\bf H}$ 
(stored as a vector). The example has both a bias offset and a
variance transform.

\mysect{Model Adaptation using MAP}{mapadapt}

Model adaptation can also be accomplished using a maximum a
posteriori (MAP) approach\index{adaptation!MAP}. 
This adaptation process is sometimes
referred to as Bayesian adaptation. MAP adaptation involves the use 
of prior knowledge about the model parameter distribution.
Hence, if we know what the parameters of the model are
likely to be (before observing any adaptation data) using the prior
knowledge, we might well be able to make good use of the limited
adaptation data, to obtain a decent MAP estimate. This type of prior
is often termed an informative prior.
Note that if the prior
distribution indicates no preference as to what the model parameters
are likely to be (a non-informative prior), then the MAP estimate
obtained will be identical to that obtained using a maximum likelihood
approach.

For MAP adaptation purposes, the informative priors that are generally
used are the speaker independent model parameters. For mathematical
tractability conjugate priors are used, which results in a simple
adaptation formula. The update formula for a 
single stream system for state $j$ and mixture component $m$ is

\hequation{
\hat{\bm{\mu}}_{jm} = \frac{ N_{jm} } { N_{jm} + \tau } \bar{\bm{\mu}}_{jm} + 
                      \frac{ \tau } { N_{jm} + \tau } \bm{\mu}_{jm}
}{meanmap}

where $\tau$ is a weighting of the a priori knowledge to the
adaptation speech data and $N$ is the occupation likelihood of the
adaptation data, defined as,
\[
   N_{jm} = \liksum{jm}
\]

where $\bm{\mu}_{jm}$ is the speaker independent mean 
and $\bar{\bm{\mu}}_{jm}$ is the mean of the observed adaptation
data and is defined as,
\[
   \bar{\bm{\mu}}_{jm} = \frac{
                \liksum{jm}\bm{o}^r_{t}}{\liksum{jm}}
\]

As can be seen, if the occupation likelihood
of a Gaussian component ($N_{jm}$) is small, then the
mean MAP estimate will remain close to the speaker
independent component mean. 
With MAP adaptation, every single mean
component in the system is updated with a MAP estimate, based on the
prior mean, the weighting and the adaptation data. Hence, MAP
adaptation requires a new ``speaker-dependent'' model set to be saved.

One obvious drawback to MAP adaptation is that it requires more
adaptation data to be effective when compared to MLLR, because MAP
adaptation is specifically defined at the component level. When
larger amounts of adaptation training data become available, MAP
begins to perform better than MLLR, due to this detailed update of
each component (rather than the pooled Gaussian transformation
approach of MLLR). In fact the two adaptation processes can be
combined to improve performance still further, by using the MLLR
transformed means as the priors for MAP adaptation (by replacing
$\bm{\mu}_{jm}$ in equation~\ref{e:meanmap} with the transformed mean
of equation~\ref{e:mtrans}). In this case
components that have a low occupation likelihood in the adaptation
data, (and hence would not change much using MAP alone) have been
adapted using a regression class transform in MLLR. An example usage
is shown in the following section. 

\pagebreak
\mysect{Using \htool{HEAdapt}}{UsingHEAdapt}

At the outset \htool{HEAdapt} operates in a very similar fashion to
\htool{HERest}. Both use a frame/state alignment
in order to accumulate various statistics about the data. In
\htool{HERest} these statistics are used to estimate new model
parameters whilst in \htool{HEAdapt} they are used to estimate 
the transformations for each regression base class, or new model
parameters. \htool{HEAdapt} will
currently only produce transforms with single stream data and
\texttt{PLAINHS} or \texttt{SHAREDHS} HMM systems (see
section~\ref{s:hmmsets} on HMM set kinds).

In outline, \htool{HEAdapt} works as follows. On startup, \htool{HEAdapt} 
loads in a complete set of HMM definitions, including, the
regression class tree and the base class number of each Gaussian
component. Note that \htool{HEAdapt} requires the MMF to contain a
regression class tree. Every training file must
have an associated label file which gives a transcription for that
file.  Only the sequence of labels is used by \htool{HEAdapt},
and any boundary location information is ignored.  Thus, these
transcriptions can be generated automatically from the known
orthography of what was said and a pronunciation dictionary.

\centrefig{headaptrdp}{120}{File Processing in HEAdapt}

\htool{HEAdapt}\index{hvite@\htool{HEAdapt}}
processes each training file in turn.
After loading it into memory, it uses the associated transcription to 
construct a  composite HMM which spans the whole utterance.
This composite HMM is made by concatenating instances of the phone HMMs 
corresponding to each label in the transcription. The Forward-Backward
algorithm is then applied to obtain a frame/state alignment and the
information  necessary to form the standard 
auxiliary function is accumulated at the Gaussian component
level. Note that this information is
different from that required in \htool{HERest} (see
section~\ref{s:mllrformulae}).
When all of the training files have been processed (within the static
or incremental block), the regression
base class statistics are accumulated using the component level
statistics. 
Next the regression class tree is traversed and the new regression class 
transformations are calculated for those regression classes containing
a sufficient occupation count at the lowest level in the tree,
as described in section~\ref{s:reg_classes}. Finally
either the updated (i.e. adapted) HMM set or the transformations are output.
Note that \htool{HEAdapt} produces a transforms model file (TMF) that 
contains transforms that are estimated to \textit{transform from the
input MMF} to a new environment/speaker based on the adaptation data  
presented.

%\index{forward-backward!embedded}

The mathematical details of the Forward-Backward algorithm are given
in section~\ref{s:bwformulae}, while the mathematical details for the