lattice-tool.1

这是一款很好用的工具包

the second operand specified by 
.BR \-in-lattice2 .
Operations currently supported are
.B concatenate
and
.BR or ,
for serial and parallel lattice combination, respectively,
and are applied after all other lattices manipulations.
.TP
.B \-viterbi-decode
Print out the word sequence corresponding to the highest probability path.
.TP
.B \-posterior-decode
Print out the word sequence with lowest expected word error.
.TP
.B \-output-ctm
Output word sequences in NIST CTM (conversation time mark) format.
Note that word start times will be relative to the lattice start time,
the first column will contain the lattice name, and the channel field
is always 1.
The word confidence field contains posterior probabilities if 
.BR \-posterior-decode
is in effect.
This option also implies
.BR \-acoustic-mesh .
.TP
.BR \-hidden-vocab " file"
Read a subvocabulary from
.I file
and constrain word meshes to only align those words that are either all
in or outside the subvocabulary.
This may be used to keep ``hidden event'' tags from aligning with
regular words.
.TP
.B \-dictionary-align
Use the dictionary pronunciations specified with
.B \-dictionary 
to induce a word distance metric used for word mesh alignment.
See the 
.BR nbest-lattice (1)
.B \-dictionary
option.
.TP
.BI \-nbest-decode " N"
Generate the up to
.I N
highest scoring paths through a lattice and write them out in 
.BR nbest-format (5),
along with optional additional score files to store knowledge sources encoded
in the lattice.
Further options are needed to specify the location of N-best lists and 
score files, described below under "N-BEST DECODING".
Duplicated Hypotheses that differ only in pause and words specified with
.B \-ignore-vocab
are removed from the N-best output.
If the
.B \-multiwords
option is specified, duplicates due to multiwords are also eliminated.
.TP
.BI \-nbest-duplicates " K"
Allow up to
.I K
duplicate word hypotheses to be output in N-best decoding.
.TP
.BI \-nbest-max-stack " M"
Limits the depth of the hypothesis stack used in N-best decoding to
.I M
entries, 
which may be useful for limiting memory use and runtime.
.TP
.B \-nbest-viterbi
Use a Viterbi algorithm to generate N-best, rather than A-star.
This uses less memory but may take more time.
.TP
.BI \-ppl " file"
Read sentences from
.I file
and compute the maximum probability (of any path) assigned to them by the
lattice being processed.
Effectively, the lattice is treated as a (deficient) language model.
The output detail is controlled by the 
.B \-debug
option, similar to
.B "ngram \-ppl"
output.
(In particular, 
.B "\-debug 2"
enables tracing of lattice nodes corresponding to sentence prefixes.)
Pause words in 
.I " file"
are treated as regular words and have to match pause nodes in the 
lattice, unless 
.B \-nopause
specified, in which case pauses in both lattice and input sentences
are ignored.
.PP
The following options control transition weight assignment:
.TP
.BI \-order " n"
Set the maximal N-gram order to be used for transition weight assignment
(the default is 3).
.TP
.BI \-lm " file"
Read N-gram language model from 
.IR file .
This option also triggers weight reassignment and lattice expansion.
.TP
.B \-multiwords
Resolve multiwords in the lattice without splitting nodes.
This is useful in rescoring lattices containing multiwords with a
LM does not use multiwords.
.TP
.BI \-classes " file"
Interpret the LM as an N-gram over word classes.
The expansions of the classes are given in
.IR file 
in 
.BR classes-format (5).
Tokens in the LM that are not defined as classes in
.I file 
are assumed to be plain words, so that the LM can contain mixed N-grams over
both words and word classes.
.TP
.BR \-simple-classes
Assume a "simple" class model: each word is member of at most one word class,
and class expansions are exactly one word long.
.TP
.BI \-mix-lm " file"
Read a second N-gram model for interpolation purposes.
The second and any additional interpolated models can also be class N-grams
(using the same
.B \-classes 
definitions).
.TP
.B \-factored
Interpret the files specified by 
.BR \-lm ,
.BR \-mix-lm ,
etc. as factored N-gram model specifications.
See 
.BR ngram (1)
for more details.
.TP
.BI \-lambda " weight"
Set the weight of the main model when interpolating with
.BR \-mix-lm .
Default value is 0.5.
.TP
.BI \-mix-lm2 " file"
.TP
.BI \-mix-lm3 " file"
.TP
.BI \-mix-lm4 " file"
.TP
.BI \-mix-lm5 " file"
.TP
.BI \-mix-lm6 " file"
.TP
.BI \-mix-lm7 " file"
.TP
.BI \-mix-lm8 " file"
.TP
.BI \-mix-lm9 " file"
Up to 9 more N-gram models can be specified for interpolation.
.TP
.BI \-mix-lambda2 " weight"
.TP
.BI \-mix-lambda3 " weight"
.TP
.BI \-mix-lambda4 " weight"
.TP
.BI \-mix-lambda5 " weight"
.TP
.BI \-mix-lambda6 " weight"
.TP
.BI \-mix-lambda7 " weight"
.TP
.BI \-mix-lambda8 " weight"
.TP
.BI \-mix-lambda9 " weight"
These are the weights for the additional mixture components, corresponding
to
.B \-mix-lm2
through
.BR \-mix-lm9 .
The weight for the
.B \-mix-lm 
model is 1 minus the sum of 
.B \-lambda
and 
.B \-mix-lambda2
through
.BR \-mix-lambda9 .
.TP
.BI \-bayes " length"
Set the context length used for Bayesian interpolation.
The default value is 0, giving the standard fixed interpolation weight
specified by
.BR \-lambda .
.TP
.BI \-bayes-scale " scale"
Set the exponential scale factor on the context likelihood in conjunction
with the
.B \-bayes
function.
Default value is 1.0.
.TP
.BI \-compact-expansion
Use a compact expansion algorithm that uses backoff nodes to reduce the 
size of expanded lattices (see paper reference below).
.TP
.BI \-old-expansion
Use older versions of the lattice expansion algorithms (both regular and
compact), that handle only trigram models and require elimination of
null and pause nodes prior to expansion.
Not recommended, but useful if full backward compatibility is required.
.TP
.BI \-max-nodes " M"
Abort lattices expansion when the number of nodes (including null and pause
nodes) exceeds 
.IR M .
This is another mechanism to avoid spending too much time on very large
lattices.
.SH "LATTICE EXPANSION ALGORITHMS"
.B lattice-tool
incorporates several different algorithms to apply LM weights to
lattices.
This section explains what algorithms are applied given what options.
.TP
.B "Compact LM expansion"
This expands the nodes and transitions to be able to assign
higher-order probabilities to transitions.
Backoffs in the LM are exploited in the expansion, thereby 
minimizing the number of added nodes (Weng et al., 1998).
This algorithm is triggered by
.BR \-compact-expansion 
For the resulting lattices to work correctly, backoff paths in the LM
must have lower weight than the corresponding higher-order paths.
(For N-gram LMs, this can be achieved using the
.B "ngram \-prune-lowprobs"
option.)
Pauses and null nodes are handled during the expansion and do
not have to be removed and restored.
.TP
.B "General LM expansion"
This expands the lattice to apply LMs of arbitrary order, 
without use of backoff transitions.
This algorithm is the default (no
.BR \-compact-expansion ).
.TP
.B "Unigram weight replacement"
This simply replaces the weights on lattice transitions with 
unigram log probabilities.
No modification of the lattice structure is required.
This algorithm is used if 
.B \-old-expansion
and
.B "\-order 1"
are specified.
.TP
.B "Bigram weight replacement"
This replaces the transition weights with bigram log probabilities.
Pause and null nodes have to be eliminated prior to the operation,
and are restored after weight replacement.
This algorithm is used if 
.B \-old-expansion
and
.B "\-order 2"
are specified.
.SH "HTK LATTICES"
.PP
.B lattice-tool
can optionally read, process, and output lattices in 
HTK Standard Lattice Format.
The following options control HTK lattice processing.
.TP
.B \-read-htk
Read input lattices in HTK format.
All lattices are internally represented as PFSGs;
to achieve this HTK lattices links
are mapped to PFSG nodes (with attached word and score information), and 
HTK lattice nodes are mapped to PFSG NULL nodes.
Transitions are created so as to preserve words and scores of all paths
through the original lattice.
On output, this mapping is reversed, so as to create a compact encoding
of PFSGs containing NULL nodes as HTK lattices.
.TP
.BI \-htk-acscale " S"
.TP
.BI \-htk-lmscale " S"
.TP
.BI \-htk-ngscale " S"
.TP
.BI \-htk-prscale " S"
.TP
.BI \-htk-duscale " S"
.TP
.BI \-htk-x1scale " S"
.TP
.BI \-htk-x2scale " S"
\&...
.TP
.BI \-htk-x9scale " S"
.TP
.BI \-htk-wdpenalty " S"
These options specify the weights for
acoustic, LM, N-gram, pronunciation, and duration models,
up to nine extra scores, as well as 
word transition penalties to be used for combining the various scores
contained in HTK lattices.
The combined scores are then used to compute the transition weights for
the internal PFSG representation.
Default weights are obtained from the specifications in the lattice files
themselves.
.br
Word transition penalties are scaled according to the log base used.
Values specified on the command line are scaled according to 
.BR \-htk-logbase ,
or the default 10.
Word transition penalties specified in the lattice file are scaled 
according to the log base specified in the file, or the default 
.IR e .
.TP
.B \-no-htk-nulls
Eliminate NULL nodes otherwise created by the conversion of HTK lattices
to PFSGs.
This creates additional links and may or may not reduce the overall
processing time required.
.TP
.BI \-dictionary " file"
Read a dictionary containing pronunciation probabilities from 
.IR file ,
and add or replace the pronunciation scores in the lattice accordingly.
This requires that the lattices contain phone alignment information.
.TP
.B \-intlogs
Assume the dictionary contains log probabilities encoded on the int-log scale,
as used by the SRI Decipher system.
.TP
.B \-write-htk
Write output lattices in HTK format.
If the input lattices were in PFSG format the original PFSG weights will be
output as HTK acoustic scores.
However, LM rescoring will discard the original PFSG weights and
the results will be encoded as LM scores.
Pronunciation scoring results will be encoded as pronunciations scores.
If the 
.B \-compute-posteriors
was used in lattice processing the output lattices will also contain
node posterior probabilities.
If the input lattices were in HTK format, then
acoustic and duration scores are preserved from the input lattices.
The score scaling factors in the lattice header will reflect the 
.B \-htk-*scale
options given above.
.TP
.BI \-htk-logbase " B"
Modify the logarithm base in HTK lattices output.
The default is to use logs base 10, as elsewhere in SRILM.
As value of 0 means to output probabilities instead of log probabilities.
Note that the log base for input lattices is not affected by this 
option; it is encoded in the lattices themselves,
and defaults to
.I e
according to the HTK SLF definition.
.TP
.B \-htk-words-on-nodes
Output word labels and other word-related information on HTK lattice nodes,
rather than links, thus saving space.
This option is provided only for compatibility with software that requires
word information to be attached specifically to links of nodes.
.TP
.B \-htk-scores-on-nodes
Output scores on HTK lattice nodes, rather than links.
.TP
Note:
The options
.BR \-no-htk-nulls ,
.BR \-htk-words-on-nodes ,
and
.BR \-htk-scores-on-nodes
defeat the mapping of internal PFSG nodes back to HTK transitions, and should
therefore NOT be used when a compact output representation is desired.
.TP
.B \-htk-quotes
Enable the HTK string quoting mechanism that allows whitespace and other
non-printable characters to be included in words labels and other fields.
This is disabled by default since PFSG lattices and other SRILM tools don't
support such word labels.
It affects both input and output format for HTK lattices.
.SH "N-BEST DECODING"
The option
.B \-nbest-decode
triggers generation of N-best lists, according to the 
aggregate score of paths encoded in the lattice.
The output format for N-best lists and associated additional score files
is compatible with other SRILM tools that process N-best lists,
such as those described in 
.BR nbest-lattice (1)
and 
.BR nbest-scripts (1).
The following options control the location of output files:
.TP
.BI \-out-nbest-dir " dir"
The directory to which N-best list files are written.
These contain acoustic model scores, language model scores,
word counts, and the word hypotheses themselves,
in SRILM format as described in
.BR nbest-format (5).
.TP
.BI \-out-nbest-dir-ngram " dir"
Output directory for separate N-gram LM scores as may be encoded in 
HTK lattices.
.TP
.BI \-out-nbest-dir-pron " dir"
Output directory for pronunciation scores encoded in HTK lattices.
.TP
.BI \-out-nbest-dir-dur " dir"
Output directory for duration model scores encoded in HTK lattices.
.TP
.BI \-out-nbest-dir-xscore1 " dir"
.TP
.BI \-out-nbest-dir-xscore2 " dir"
\&...
.TP
.BI \-out-nbest-dir-xscore9 " dir"
Output score directories for up to nine additional knowledge sources
encoded in HTK lattices.
.TP
.BI \-out-nbest-dir-rttm " dir"
N-best hypotheses in NIST RTTM format.
This function is experimental and makes assumptions about the input 
file naming conventions to infer timing information.
.SH "SEE ALSO"
ngram(1), pfsg-scripts(1), nbest-lattice(1),
pfsg-format(5), ngram-format(5), classes-format(5), wlat-format(5),
nbest-format(5).
.br
F. Weng, A. Stolcke, and A. Sankar,
``Efficient Lattice Representation and Generation.''
\fIProc. Intl. Conf. on Spoken Language Processing\fP, vol. 6, pp. 2531\-2534,
Sydney, 1998.
.br
S. Young et al., \fIThe HTK Book\fP, HTK version 3.1.
http://htk.eng.cam.ac.uk/prot-docs/htk_book.shtml
.SH BUGS
Not all LM types supported by 
.BR ngram (1)
are handled by 
.B lattice-tool.
.PP
Care must be taken when processing multiword lattices with 
.B \-unk 
and 
.B \-multiwords
or 
.BR \-split-multiwords .
Multiwords not listed in the LM (or the explicit vocabulary specified) will
be considered ``unknown'', even though their components might be 
in-vocabulary.
.PP
The 
.B \-nbest-duplicates
option does not work together with
.BR \-nbest-viterbi .
.PP
Input lattice in HTK format may contain node or link posterior information.
However, this information is effectively discarded; posteriors are always
recomputed from scores when needed for pruning or output.
.PP
The 
.BR \-no-nulls ,
.B \-no-pause
and
.B \-compact-pause 
options discard the acoustic information associated with NULL and pause
nodes in HTK lattice input, and should therefore not be used if 
equivalent HTK lattice output is intended.
.SH AUTHORS
Fuliang Weng <[e-mail]>
.br
Andreas Stolcke <[e-mail]>
.br
Dustin Hillard <[e-mail]>
.br
Jing Zheng <[e-mail]>
.br
Copyright 1997\-2006 SRI International