refine.tex

隐马尔科夫模型工具箱

2 states from each of the \texttt{aa} models and 3 states from
each of the others.  Moving further down the tree, the item list
\begin{verbatim}
     { *.state[2-4].stream[1].mix[1,3].cov }
\end{verbatim}
denotes the set of all covariance vectors (or matrices) of the first and
third mixture
components of stream 1, of states 2 to 4 of all HMMs.  Since many HMM systems
are single stream, the \texttt{stream} part of the path can be omitted if its value
is 1.  Thus, the above could have been written
\begin{verbatim}
     { *.state[2-4].mix[1,3].cov }
\end{verbatim}
These last two examples also show that indices\index{item lists!indexing} can be written as comma
separated lists as well as ranges, for example, \texttt{[1,3,4-6,9]}
is a valid index list representing states 1, 3, 4, 5, 6, and 9.

When item lists are used as the argument to a \texttt{TI} 
command\index{ti@\texttt{TI} command}, the
kind of items represented by the list determines the macro type in a fairly
obvious way.  The only non-obvious cases are firstly that lists ending
in \texttt{cov} generate \hmmt{v}, \hmmt{i}, \hmmt{c}, or \hmmt{x} macros as
appropriate.   If an explicit set of mixture components is defined
as in
\begin{verbatim}
     { *.state[2].mix[1-5] }
\end{verbatim}
then  \hmmt{m} macros are generated but omitting
the indices  altogether denotes a special case of mixture 
tying\index{tied-mixtures}
which is explained later in Chapter~\ref{c:discmods}.

To illustrate the use of item lists, some example \texttt{TI} commands
can now be given.  Firstly, when a set of context-dependent models is created, it can
be beneficial to share one transition matrix across all variants
of a phone rather than having a distinct transition matrix for each.
This could be achieved by adding \texttt{TI}
commands immediately after the \texttt{CL} command described in
the previous section, that is\index{tying!examples of}
\begin{verbatim}
    CL cdlist
    TI T_ah {*-ah+*.transP}
    TI T_eh {*-eh+*.transP}
    TI T_ae {*-ae+*.transP}
    TI T_ih {*-ih+*.transP}
     ... etc
\end{verbatim}

As a second example, a so-called Grand Variance\index{grand variance} 
HMM system can
be generated very easily with the following HHEd command
\begin{verbatim}
     TI "gvar" { *.state[2-4].mix[1].cov }
\end{verbatim}
where it is assumed that the HMMs are 3-state 
single mixture component models.   The effect
of this command is to tie all state distributions to a single global variance
vector.  For applications, where there is limited training data, this technique
can improve performance, particularly in noise.

Speech recognition systems will often have distinct
models for silence  and short pauses.  A 
silence model\index{silence model} \texttt{sil} may have
the normal 3 state topology whereas a short pause model may have just 
a single state.  To avoid the two models \textit{competing} with each other, the
\texttt{sp} model state can be tied to the centre state of the \texttt{sil} model
thus
\begin{verbatim}
     TI "silst" { sp.state[2], sil.state[3] }
\end{verbatim}

So far nothing has been said about how the parameters are actually
determined when a set of items is replaced by a single shared representative.
When states are tied, the state with the broadest  variances  and as few as
possible zero mixture component weights is selected from the pool and used
as the representative.  When mean vectors are tied, the average of all the
mean vectors in the pool is used and when variances are tied, the largest
variance in the the pool is used.  In all other cases, the last item in the
tie-list is  arbitrarily chosen as representative.
All of these selection criteria are \textit{ad hoc}, but since
the tie operations are always followed by explicit re-estimation
using \htool{HERest}, the precise choice of representative for a tied
set is not critical.\index{tying!exemplar selection}

Finally, tied parameters can be
untied.  For example,  subsequent refinements of the context-dependent model set
generated above with tied transition matrices might result in
a much more compact set of models for which individual transition
parameters could be robustly estimated.    This 
can be done using the \texttt{UT} command\index{ut@\texttt{UT} command} whose effect is to untie all of the
items in its argument list.  For example, the command
\begin{verbatim}
     UT {*-iy+*.transP}
\end{verbatim}
would untie the transition parameters in all variants of the \texttt{iy}
phoneme.
This untying works by simply making unique copies of the tied parameters.
These untied parameters can then subsequently be re-estimated.

\mysect{Data-Driven Clustering}{ddclust}
 
In
section~\ref{s:mkCDHMMs}, a method of triphone construction was described
which involved cloning all monophones and then re-estimating them using data
for which monophone labels have been replaced by triphone labels.  
This will lead to a very large set of models, and relatively little
training data for each model.  Applying the argument that context will not greatly affect
the centre states of triphone models, one way to reduce the 
total number of parameters without 
significantly altering the models' ability to represent the different
contextual effects might be to tie all of the centre states across all
models derived from the same monophone.  This tying could 
be\index{clustering!data-driven}
done by writing an edit script of the form
\begin{verbatim}
     TI "iyS3" {*-iy+*.state[3]}
     TI "ihS3" {*-ih+*.state[3]}
     TI "ehS3" {*-eh+*.state[3]}
      .... etc
\end{verbatim}
Each \texttt{TI} command would tie all the centre states of all triphones
in each phone group. Hence, if there were an average of 100 triphones
per phone group then the total number of states per group
would be reduced from
300 to 201.

Explicit tyings such as these can have some positive effect but overall they 
are not very satisfactory.  Tying all centre states is too severe and worse
still, the problem of undertraining for the left and right states remains.
A much better approach is to use clustering to decide which states to
tie.  \htool{HHEd} provides two mechanisms for this.  In this section
a data-driven clustering approach will be described and in
the next section, an alternative decision tree-based approach is presented.

Data-driven clustering is performed by the \index{model training!clustering}
\texttt{TC}\index{tc@\texttt{TC} command} and 
\texttt{NC}\index{nc@\texttt{NC} command}
commands.  These both invoke the same top-down hierarchical
procedure.  Initially all states are placed in individual
clusters.  The pair of clusters which when combined would form the smallest
resultant cluster are merged.  This process repeats until either the
size of the largest
cluster reaches the threshold set by the \texttt{TC} command or
the total number of clusters has fallen to that
specified by by the \texttt{NC} command.  The size of cluster
is defined as the greatest distance between any two states.
The distance metric depends on the type of state distribution.
For single Gaussians, a weighted Euclidean distance between the means
is used and for tied-mixture systems a  Euclidean distance between the
mixture weights is used.  For all other cases, the average probability
of each component mean with respect to the other state is used.
The details of the algorithm and these metrics are given in the reference
section for \htool{HHEd}.

\centrefig{tiedstate}{100}{Data-driven state tying}

As an example, the following \htool{HHEd} script would cluster and tie the
corresponding states of the triphone group for the phone \texttt{ih}
\begin{verbatim}
     TC 100.0 "ihS2" {*-ih+*.state[2]}
     TC 100.0 "ihS3" {*-ih+*.state[3]}
     TC 100.0 "ihS4" {*-ih+*.state[4]}
\end{verbatim}
In this example, each \texttt{TC} command performs clustering on the specified
set of states, each cluster is  tied and output as a macro.  The macro name
is generated by appending the cluster index to
the macro  name given in the command.   The effect of this command 
is illustrated in Fig.~\href{f:tiedstate}.  Note that if a word-internal
triphone system is being built, it is sensible to include biphones as well
as triphones in the item list, for example, the first command above would
be written as
\begin{verbatim}
     TC 100.0 "ihS2" {(*-ih,ih+*,*-ih+*).state[2]}
\end{verbatim}
If the above \texttt{TC} commands are repeated for all phones, the resulting  set of
tied-state models will have far
fewer parameters in total than the original untied set.  The numeric argument
immediately following the \texttt{TC} command name is the cluster threshold.  Increasing
this value will allow larger and hence, fewer clusters. The aim, of
course, is to strike the right balance between compactness and the acoustic
accuracy of the individual models.  In practice, the use of this command
requires some experimentation to find a good threshold value. \htool{HHEd} provides
extensive trace  output for monitoring clustering operations.  Note in this
respect that as well as setting tracing from the command line and the
configuration file, tracing in \htool{HHEd} can be set by the \texttt{TR} command. 
Thus,  tracing can be controlled at the command level. Further trace
information can be obtained by including the \texttt{SH} command\index{sh@\texttt{SH} command} at strategic
points in the edit script.  The effect of executing this command is to list
out all of the parameter tyings currently in force.

A potential problem with the use of the \texttt{TC} and \texttt{NC} commands is
that {\it outlier} states will tend to form their own singleton 
clusters\index{singleton clusters} for
which there is then insufficient data to properly train.  One solution to
this is to use the \texttt{RO} command\index{ro@\texttt{RO} command} to 
remove outliers\index{removing outliers}.  This commmand has
the form
\begin{verbatim}
     RO thresh "statsfile"
\end{verbatim}
where \texttt{statsfile} is the name of a statistics file\index{statistics
file} output using the
\texttt{-s} option of \htool{HERest}.  This statistics file holds the 
{\em occupation counts} for all states of the HMM set being trained.  
The term {\em occupation count} refers to the number of frames allocated to a
particular state and can be used as a measure of how much training data is
available for estimating the parameters of that state.  
The \texttt{RO} command must be executed {\it before} the \texttt{TC} or
\texttt{NC} commands used to do the actual clustering. Its effect is to simply
read in the statistics information from the given file and then to set a flag
instructing the
\texttt{TC} or \texttt{NC} commands to remove any outliers remaining at 
the conclusion
of the normal clustering process.  This is done by repeatedly finding the
cluster with the smallest total occupation count and merging it with its
nearest neighbour. This process is repeated until all clusters have a total
occupation count which exceeds \texttt{thresh}, thereby ensuring that every
cluster of states will be properly trained in the subsequent re-estimation
performed by \htool{HERest}.\index{state tying}


On completion of the above clustering and tying procedures, many of the models
may be effectively identical, since acoustically similar triphones may share
common clusters for all their emitting states.  They are then, in effect,
so-called {\it generalised triphones}.\index{generalised triphones} State tying
can be further exploited if the HMMs which are effectively equivalent are
identified and then tied via the physical-logical mapping\footnote{The physical
HMM which corresponding to several logical HMMs will be arbitrarily named after
one of them.} facility provided by HMM lists (see section~\ref{s:hmmsets}). The
effect of this would be to reduce the total number of HMM definitions required.
\htool{HHEd} provides a compaction command to do all of this automatically.
For example, the command
\begin{verbatim}
     CO newList 
\end{verbatim}
\index{co@\texttt{CO} command}will compact\index{model training!compacting} 
the currently loaded HMM set by identifying equivalent models
and then tying them via the new HMM list output  to the 
file \texttt{newList}.  Note, however, that for two HMMs to be tied, they
must be identical in all respects.
This is one of the reasons why transition parameters are often tied
across triphone groups otherwise HMMs with identical states would still
be left distinct due to minor differences in their transition matrices.

\mysect{Tree-Based Clustering}{tbclust}

\index{clustering!tree-based}
One limitation of the data-driven clustering procedure described above is
that it does not deal with triphones for which there are no examples in the
training data.  When building word-internal triphone systems,  this 
problem can often
be avoided by careful design of the training database but when building large
vocabulary cross-word triphone systems \textit{unseen} triphones are unavoidable.
\index{unseen triphones}

\centrefig{qstree}{100}{Decision tree-based state tying}

\htool{HHEd} provides an alternative decision 
tree based clustering\index{decision tree-based clustering} mechanism
which provides a similar quality of clustering but offers a solution 
to the unseen triphone problem.  Decision tree-based clustering is invoked
by the command \texttt{TB} which is analogous to the \texttt{TC} command
described above and has an identical form, that is
\begin{verbatim}