labels.tex

隐马尔科夫模型工具箱

the MLF then that is loaded, otherwise the file \texttt{f} is opened as normal.
If \texttt{f} does not exist, then an error is reported.
The \texttt{-I} option may be repeated on the command line to open
several MLF files simultaneously.  In this case, each is searched in turn
before trying to open the required file.

MLFs can do two things.  Firstly, they can contain 
embedded label definitions\index{master label files!embedded label definitions} 
so that many or all of the needed label definitions can be
stored in the same file.  Secondly, they can contain the names of
sub-directories to search for label files.  In effect, they allow
multiple search paths\index{master label files!multiple search paths} 
to be defined.  Both of these two types of
definition can be mixed in a single MLF.

MLFs are quite complex to understand and use.  However, they add
considerable power and flexibility to \HTK\ which combined with the
\texttt{-S}\index{standard options!aaas@\texttt{-S}} and 
\texttt{-L}\index{standard options!aaal@\texttt{-L}} 
options mean that virtually any organisation of
data and label files can be accommodated. 

\subsection{Syntax and Semantics}

An MLF consists of one or more individual definitions.  Blank lines in
an MLF are ignored but otherwise the line structure is significant.
The first line must contain just \texttt{\#!MLF!\#} to identify it as an MLF file.
This is not necessary for use with the \texttt{-I} option but some 
\HTK\ tools need to be able to distinguish an MLF from a normal label file.
The following syntax\index{master label files!syntax} of MLF files is described using an 
extended BNF notation in which
alternatives are separated by a vertical bar $|$, parentheses (\ ) denote
factoring, brackets [\ ] denote options, and braces \{\ \} denote zero or more
repetitions. 

{\sf
\begin{tabbing}
++++ \= ++++++++ \= ++ \= +++++++++++++++++ \= +++ \=  \kill
\>    MLF  =  \> ``\#!MLF!\#'' \\
\>\>  MLFDef \{ MLFDef \}
\end{tabbing}
}

Each definition is either a transcription for immediate loading or a
subdirectory to search.

{\sf
\begin{tabbing}
++++ \= ++++++++ \= ++ \= +++++++++++++++++ \= +++ \=  \kill
\>    MLFDef  = \> ImmediateTranscription $|$ SubDirDef
\end{tabbing}
}

An immediate transcription consists of a pattern on a line
by itself immediately followed
by a transcription which as far as the MLF is concerned is arbitrary
text.  It is read using whatever label file ``driver'' routines are
installed in \htool{HLabel}.  It is terminated by a period written on a line
of its own.

{\sf
\begin{tabbing}
++++ \= ++++++++ \= ++ \= +++++++++++++++++ \= +++ \=  \kill
\>   ImmediateTranscription = \\
\>\>                    Pattern \\
\>\>                    Transcription \\
\>\>``.'' 
\end{tabbing}
}
A subdirectory definition  simply gives the name of a subdirectory
to search.  If the required label file is found in that subdirectory
then the label file is loaded, otherwise the next matching subdirectory
definition is checked.

{\sf
\begin{tabbing}
++++ \= ++++++++ \= ++ \= +++++++++++++++++ \= +++ \=  \kill
\>   SubDirDef = \> Pattern SearchMode String \\
\>   SearchMode = \> ``\verb+->+'' $|$ ``\verb+=>+''
\end{tabbing}
}
\noindent
The two types of search mode are described below. A pattern is just a 
string
{\sf
\begin{tabbing}
++++ \= ++++++++ \= ++ \= +++++++++++++++++ \= +++ \=  \kill
\>    Pattern = \> String
\end{tabbing}
}
\noindent
except that the characters `\texttt{?}' and `\texttt{*}' embedded in the string act
as wildcards\index{master label files!wildcards} such that `\texttt{?}' matches any single character and
`\texttt{*}' matches 0 or more characters.
A string is any sequence of characters enclosed in double quotes.

\subsection{MLF Search}

The names of label files in \HTK\ are invariably reconstructed from an
existing data file name and this means that the
file names used to access label files can be partial or full path names in which
the path has been constructed either from the path of the corresponding
data file or by direct specification via the {\tt -L} option.  These
path names are retained in the MLF search\index{master label files!search} which proceeds as follows.
The given label file specification \texttt{../d3/d2/d1/name} is matched
against each pattern in the MLF.  If a pattern matches, then either the
named subdirectory is searched or an immediate definition is loaded.
Pattern matching continues in this way until a definition is found.  If
no pattern matches\index{master label files!pattern matching} then an attempt is made to open \texttt{../d3/d2/d1/name}
directly.  If this fails an error is reported.

The search of a sub-directory\index{master label files!sub-directory search} proceeds as follows.  In simple search
mode indicated by  \texttt{->}, the file \texttt{name} must occur directly in the
sub-directory.  In full search mode indicated by \texttt{=>}, the files
\texttt{name, d1/name, d2/d1/name}, etc. are searched for in that order.
This full search allows a hierarchy of label files to be constructed
which mirrors a hierarchy of data files (see Example 4 below).
   
Hashing is performed when the label file specification is either
a full path name or in the form \texttt{*/file} so in these cases
the search is very fast. Any other use of metacharacters invokes
a linear search with a full and relatively slow pattern match at each step.
Note that all tools which generate label files have a \texttt{-l}
option which is used to define the output directory in which to store
individual label files.  When outputting master label files,  the \texttt{-l}
option can be used to define the path in the output label file specifications.
In particular, setting the option \texttt{-l '*'} causes
the form \texttt{*/file} to be generated.\index{master label files!patterns}

\subsection{MLF Examples}

\index{master label files!examples}
\begin{enumerate}
\item
Suppose a data set consisted of two training data files with
corresponding label files:
\newline
\texttt{a.lab} contains 
\begin{verbatim}
       000000  590000  sil
       600000 2090000  a
      2100000 4500000  sil
\end{verbatim}
\texttt{b.lab} contains 
\begin{verbatim}
       000000  990000  sil
      1000000 3090000  b
      3100000 4200000  sil
\end{verbatim}
Then the above two individual label files could be replaced by a single MLF
\begin{verbatim}
      #!MLF!#
      "*/a.lab"
       000000  590000  sil
       600000 2090000  a
      2100000 4500000  sil
      .                      
      "*/b.lab"
       000000  990000  sil
      1000000 3090000  b
      3100000 4200000  sil
      .                      
\end{verbatim}

\item
A digit data base contains training tokens \texttt{one.1.wav, one.2.wav, one.3.wav, ...,
two.1.wav, two.2.wav, two.3.wav, ...}, etc.  Label files are required containing
just the name of the model so that \HTK\ tools such as \htool{HERest} can be used.
If MLFs are not used, individual label files are needed.  For example,
the individual label files \texttt{one.1.lab, one.2.lab, one.3.lab, ....} would be
needed
to identifiy instances of ``one'' even though each file contains the same entry, just
\begin{verbatim}
      one
\end{verbatim}
Using an MLF containing
\begin{verbatim}
      #!MLF!#
      "*/one.*.lab"
      one
      .
      "*/two.*.lab"
      two
      .
      "*/three.*.lab"
      three
      .
      <etc.>
\end{verbatim}
avoids the need for many duplicate label files.

\item
A training database \texttt{/db} contains directories \texttt{dr1, dr2, ..., dr8}.
Each directory contains a subdirectory called \texttt{labs} holding the
label files for the data files in that directory.  The following
MLF would allow them to be found
\begin{verbatim}
      #!MLF!#
      "*" -> "/db/dr1/labs"
      "*" -> "/db/dr2/labs"
      ...
      "*" -> "/db/dr7/labs"
      "*" -> "/db/dr8/labs"
\end{verbatim}      
Each attempt to open a label file will result in a linear search
through \texttt{dr1} to \texttt{dr8} to find that file.  If the sub-directory name
is embedded into the label file name, then this searching can
be avoided.  For example, if the label files in directory \texttt{drx} had
the form \texttt{drx\_xxxx.lab}, then the MLF would be written as
\begin{verbatim}
      #!MLF!#
      "*/dr1_*" -> "/db/dr1/labs"
      "*/dr2_*" -> "/db/dr2/labs"
      ...
      "*/dr7_*" -> "/db/dr7/labs"
      "*/dr8_*" -> "/db/dr8/labs"
\end{verbatim}   

\item
A training database is organised as a hierarchy where 
\texttt{/disk1/db/dr1/sp2/u3.wav} is the data file for the third
repetition from speaker 2 in dialect region \texttt{dr1}
(see Figure~\ref{f:dbhier}).  

\centrefig{dbhier}{80}{Database Hierarchy: Data [Left]; Labels [Right].} 

Suppose
that a similar hierarchy of label files was constructed on
\texttt{disk3}.
These label files could be found by any \HTK\ tool by using an
MLF containing just
\begin{verbatim}
      #!MLF!#
      "*" => "/disk3"
\end{verbatim}      
If for some reason all of the \texttt{drN} directories were
renamed \texttt{ldrN} in the label hierarchy, then this could be
handled by an MLF file containing
\begin{verbatim}
      #!MLF!#
      "*/dr1/*" => "/disk3/ldr1"
      "*/dr2/*" => "/disk3/ldr2"
      "*/dr3/*" => "/disk3/ldr3"
               etc.
\end{verbatim}      
\end{enumerate}
These few examples should illustrate the flexibility and power of MLF files.
It should noted, however, that when generating label names automatically from data file names, HTK sometimes discards path details. For example, during recognition, if the data files /disk1/dr2/sx43.wav and /disk2/dr4/sx43.wav are being recognised, and a single directory is specified for the output label files, then recognition results for both files will be written to a file called sx43.lab, and the latter occurrence will overwrite the former.


\mysect{Editing Label Files}{edlab}

\HTK\ training tools typically expect the labels used in 
transcription files to correspond\index{labels!editing}
directly to the names of the HMMs chosen to build an application.  Hence,
the label files supplied with a speech database will often need 
modifying.  For example, the original transcriptions attached to a database
might be at a fine level of acoustic detail.  Groups of labels corresponding 
to a sequence of acoustic events (e.g. \texttt{pcl p'}) might need converting
to some simpler form (e.g. \texttt{p}) which is more suitable for being
represented by a HMM.  As a second example, current high performance speech
recognisers use a large number of context dependent models to allow more
accurate acoustic modelling.  For this case, the labels in the transcription
must be converted to show the required contexts explicitly.

\HTK\ supplies a tool called \htool{HLEd} for rapidly and efficiently converting
label files. The \htool{HLEd} command invocation specifies the names of the files
to be converted and the name of a script file holding the actual
\htool{HLEd}\index{hled@\htool{HLEd}} commands.  For example, the command
\begin{verbatim}
    HLEd edfile.led l1 l2 l3
\end{verbatim}
would apply the edit commands stored in the file \texttt{edfile.led}
to each of the label files \texttt{l1}, \texttt{l2} and \texttt{l3}. More commonly
the new label files are stored in a new directory to avoid overwriting