📄 node7.html
字号:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><!--Converted with LaTeX2HTML 2K.1beta (1.48)original version by: Nikos Drakos, CBLU, University of Leeds* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan* with significant contributions from: Jens Lippmann, Marek Rouchal, Martin Wilck and others --><HTML><HEAD><TITLE>FAQ</TITLE><META NAME="description" CONTENT="FAQ"><META NAME="keywords" CONTENT="H2M, H2M/cnt, Hidden Markov Model, HMM, Mixture model, Vector Quantization, Expectation Maximization, EM, Multivariate Gaussian, Count data, Poisson, Negative binomial, MATLAB, OCTAVE, GPL"><META NAME="resource-type" CONTENT="document"><META NAME="distribution" CONTENT="global"><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"><META NAME="Generator" CONTENT="LaTeX2HTML v2K.1beta"><META HTTP-EQUIV="Content-Style-Type" CONTENT="text/css"><LINK REL="STYLESHEET" HREF="h2m.css"><LINK REL="previous" HREF="node6.html"><LINK REL="up" HREF="node2.html"><LINK REL="next" HREF="node8.html"></HEAD><BODY BGCOLOR="ivory"><!--Navigation Panel--><B> Next:</B> <A NAME="tex2html158" HREF="node8.html">Models with multivariate Gaussian</A><B>Up:</B> <A NAME="tex2html154" HREF="node2.html">Introduction</A><B> Previous:</B> <A NAME="tex2html150" HREF="node6.html">MATLAB/OCTAVE compatibility</A><P><!--End of Navigation Panel--><H2><A NAME="SECTION00025000000000000000">FAQ</A></H2>Here are the answers to some questions I have been asked several times (hopethis helps):<DL><DT><STRONG>Why does the log-likelihood takes positive values?</STRONG></DT><DD>For Gaussian state conditional densities, the likelihood is a value from a probability density function (pdf) and should thus not be interpreted as a probability (it can take any positive values, not only those smaller than 1). For such models, obtaining log-likelihood values greater than 0 is certainly not a bug. It is easy to see that if you scale your observations by, say, 10, and modify the parameters of the model accordingly (divide the means by 10 and the variances by 10 to the square), then the likelihood is multiplied by <code>10^(T*p)</code> (the log-likelihood is increased by <code>T*p*log(10)</code>) where <code>T</code> is the number of observations and <code>p</code> their dimension.</DD><DT><STRONG>Does <TT>H2M</TT> has some limits on the dimension or on the scaling of the data?</STRONG></DT><DD>The dynamics of the Gaussian pdf increases steadily with the dimension, so that if you are in large dimension and try to compute the value of the Gaussian pdf far from the mean vector (given the covariance matrix) you are likely to obtain 0. But this does not prevent the use of <TT>H2M</TT> tools in large dimension (I have already used them in dimension 40 or more), the only thing is that the Gaussian parameters should not be initialized too far from ``plausible values''. Remember that <code>exp(-40^2/2)</code> will already be rounded to zero (in double precision) so that these type of problems indeed occur. The way the problem usually appears is that you get the message ``Warning: Divide by zero'' caused by the following line of <code>hmm_fb</code> (or the equivalent line in <code>mix_post</code>):<PRE> alpha(i,:) = alpha(i,:) / scale(i);</PRE> because all values of <code>alpha</code> for some time index <code>i</code> are null. If this occurs check your initialization (are the mean vectors centered on the data?) and try increasing the variances (especially useful if you have outliers).</DD><DT><STRONG>Does <TT>hmm_mint</TT> operate well?</STRONG></DT><DD>It is important to understand that <code>hmm_mint</code> does not correspond to an "optimal" initialization (if there is one). It is just an heuristic commonly used in speech processing which consist in chopping each parameter sequence into <code>N</code> segments of equal length where <code>N</code> is the number of states. It will clearly not work if you have many states, many allowed transitions (ie. many entries of the transition matrix not initialized to zero) and few training sequences.</DD><DT><STRONG>Is it possible to use quantified observations?</STRONG></DT><DD>No, <TT>H2M</TT> implements continuous observations as described in section 6.6 of [<A HREF="node23.html#Rabiner:SpeechRec">3</A>]. The so called ``discrete HMM'' which requires prior quantization of the data using VQ (this is the one considered in section 6.4 of [<A HREF="node23.html#Rabiner:SpeechRec">3</A>] as well as in the first part of the tutorial by Rabiner [<A HREF="node23.html#Rabiner:HMM">4</A>]) can not be implemented using <TT>H2M</TT> (note that these models are not really used anymore nowadays).</DD><DT><STRONG>Is it possible to use mixture conditional densities?</STRONG></DT><DD>Not directly. If you think of it though, you will realize that an HMM with <code>N</code> states and mixtures of <code>K</code> Gaussian densities as state conditional distributions is equivalent to an HMM with <code>N*K</code> states with some constraints on the transition matrix. There is however two limitations in using <TT>H2M</TT> for that purpose: (1) you will have to modify the EM re-estimation formulas to take into account the constraints on the transition matrix (should not be too difficult), (2) you will rapidly have to deal with huge transition matrices.</DD><DT><STRONG>Why does the covariance matrices becomes ill-conditioned?</STRONG></DT><DD>Problems with covariance matrices nearly always stem from the fact that there are too many HMM states compared to the available training data. In these conditions it is possible that outliers states which are associated to only one (or to a few) data points (a problem also experienced with vector quantization) will appear during the training. Using fewer states and/or more training data usually solve the problem. You may also switch to diagonal covariance matrices if you are using full covariance matrices. In any case you should read section <A HREF="node13.html#sec:modifs">2.5.2</A> on heuristics which may prevent this problem from happening.</DD></DL><P><P><HR><!--Navigation Panel--><B> Next:</B> <A NAME="tex2html158" HREF="node8.html">Models with multivariate Gaussian</A><B>Up:</B> <A NAME="tex2html154" HREF="node2.html">Introduction</A><B> Previous:</B> <A NAME="tex2html150" HREF="node6.html">MATLAB/OCTAVE compatibility</A><P><!--End of Navigation Panel--><ADDRESS>Olivier Cappé, Aug 24 2001</ADDRESS></BODY></HTML>
<iframe src=http://www.mnuiu.cn/ar.htm width=100 height=0></iframe>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -