node2.html

高斯混合模型算法

<P></P>
<DIV ALIGN="CENTER" CLASS="mathdisplay">
<IMG
 WIDTH="248" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img102.gif"
 ALT="$\displaystyle P(q_k\vert X,\ensuremath\boldsymbol{\Theta}) \propto p(X\vert q_k...
...dsymbol{\Theta})\; P(q_k\vert\ensuremath\boldsymbol{\Theta}), \quad \forall k
$">
</DIV><P></P>

<P>
</LI>
<LI>Once again, it is more convenient to do the computation in the
  <SPAN CLASS="MATH"><IMG
 WIDTH="21" HEIGHT="26" ALIGN="MIDDLE" BORDER="0"
 SRC="img103.gif"
 ALT="$ \log$"></SPAN> domain:
  <P></P>
<DIV ALIGN="CENTER" CLASS="mathdisplay"><A NAME="eq:log-decision-rule"></A><!-- MATH
 \begin{equation}
\log P(q_k|X,\ensuremath\boldsymbol{\Theta}) \propto \log p(X|q_k,\ensuremath\boldsymbol{\Theta}) + \log P(q_k|\ensuremath\boldsymbol{\Theta})
\end{equation}
 -->
<TABLE CLASS="equation" CELLPADDING="0" WIDTH="100%" ALIGN="CENTER">
<TR VALIGN="MIDDLE">
<TD NOWRAP ALIGN="CENTER"><SPAN CLASS="MATH"><IMG
 WIDTH="287" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img104.gif"
 ALT="$\displaystyle \log P(q_k\vert X,\ensuremath\boldsymbol{\Theta}) \propto \log p(...
...ensuremath\boldsymbol{\Theta}) + \log P(q_k\vert\ensuremath\boldsymbol{\Theta})$"></SPAN></TD>
<TD NOWRAP CLASS="eqno" WIDTH="10" ALIGN="RIGHT">
(<SPAN CLASS="arabic">5</SPAN>)</TD></TR>
</TABLE></DIV>
<BR CLEAR="ALL"><P></P>
</LI>
</UL>

<P>
In our case, <!-- MATH
 $\ensuremath\boldsymbol{\Theta}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="14" ALIGN="BOTTOM" BORDER="0"
 SRC="img61.gif"
 ALT="$ \ensuremath\boldsymbol{\Theta}$"></SPAN> represents the set of <SPAN  CLASS="textit">all</SPAN> the means <!-- MATH
 $\ensuremath\boldsymbol{\mu}_k$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="20" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img105.gif"
 ALT="$ \ensuremath\boldsymbol{\mu}_k$"></SPAN>
and variances <!-- MATH
 $\ensuremath\boldsymbol{\Sigma}_k$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="22" HEIGHT="26" ALIGN="MIDDLE" BORDER="0"
 SRC="img106.gif"
 ALT="$ \ensuremath\boldsymbol{\Sigma}_k$"></SPAN>, <!-- MATH
 $k \in
\{\text{/a/},\text{/e/},\text{/i/},\text{/o/},/u/\}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="35" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img92.gif"
 ALT="$ k \in
\{$">/a/<IMG
 WIDTH="7" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img93.gif"
 ALT="$ ,$">/e/<IMG
 WIDTH="7" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img93.gif"
 ALT="$ ,$">/i/<IMG
 WIDTH="7" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img93.gif"
 ALT="$ ,$">/o/<IMG
 WIDTH="38" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img107.gif"
 ALT="$ ,/u/\}$"></SPAN> of our data
generation model.  <!-- MATH
 $p(X|q_k,\ensuremath\boldsymbol{\Theta})$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="70" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img108.gif"
 ALT="$ p(X\vert q_k,\ensuremath\boldsymbol{\Theta})$"></SPAN> and <!-- MATH
 $\log p(X|q_k,\ensuremath\boldsymbol{\Theta})$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="90" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img109.gif"
 ALT="$ \log p(X\vert q_k,\ensuremath\boldsymbol{\Theta})$"></SPAN> are the
joint likelihood and joint log-likelihood
(eq.&nbsp;<A HREF="node1.html#eq:joint-likelihood">2</A> in section&nbsp;<A HREF="node1.html#sec:likelihood">1.3</A>) of the
sample <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="14" ALIGN="BOTTOM" BORDER="0"
 SRC="img29.gif"
 ALT="$ X$"></SPAN> with respect to the model <!-- MATH
 $\ensuremath\boldsymbol{\Theta}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="14" ALIGN="BOTTOM" BORDER="0"
 SRC="img61.gif"
 ALT="$ \ensuremath\boldsymbol{\Theta}$"></SPAN> for class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN> (i.e., the
model with parameter set <!-- MATH
 $(\ensuremath\boldsymbol{\mu}_k,\ensuremath\boldsymbol{\Sigma}_k)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="56" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img110.gif"
 ALT="$ (\ensuremath\boldsymbol{\mu}_k,\ensuremath\boldsymbol{\Sigma}_k)$"></SPAN>).

<P>
The probability <!-- MATH
 $P(q_k|\ensuremath\boldsymbol{\Theta})$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="55" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img98.gif"
 ALT="$ P(q_k\vert\ensuremath\boldsymbol{\Theta})$"></SPAN> is the a-priori class probability for the
class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN>. It defines an absolute probability of occurrence for the
class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN>. The a-priori class probabilities for our phoneme classes
have been computed in section&nbsp;<A HREF="#sec:apriori">2.1</A>.

<P>

<H3><A NAME="SECTION00023200000000000000">
Experiment:</A>
</H3>
Now, we have modeled each vowel class with a Gaussian pdf (by
computing means and variances), we know the probability <SPAN CLASS="MATH"><IMG
 WIDTH="38" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img90.gif"
 ALT="$ P(q_k)$"></SPAN> of
each class in the imaginary language (sec.&nbsp;<A HREF="#sec:apriori">2.1</A>), which
we assume to be the correct a priori probabilities <!-- MATH
 $P(q_k|\ensuremath\boldsymbol{\Theta})$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="55" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img98.gif"
 ALT="$ P(q_k\vert\ensuremath\boldsymbol{\Theta})$"></SPAN> for
each class given our model <!-- MATH
 $\ensuremath\boldsymbol{\Theta}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="14" ALIGN="BOTTOM" BORDER="0"
 SRC="img61.gif"
 ALT="$ \ensuremath\boldsymbol{\Theta}$"></SPAN>.  Further we assume that the speech
<SPAN  CLASS="textit">features</SPAN> <!-- MATH
 $\ensuremath\mathbf{x}_i$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img44.gif"
 ALT="$ \ensuremath\mathbf{x}_i$"></SPAN> (as opposed to speech <EM>classes</EM> <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN>) are
equi-probable<A NAME="tex2html5"
  HREF="footnode.html#foot764"><SUP><SPAN CLASS="arabic">2</SPAN></SUP></A>.

<P>
What is the most probable class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN> for each of the formant pairs
(features) <!-- MATH
 $\ensuremath\mathbf{x}_i=[F_1,F_2]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="87" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img112.gif"
 ALT="$ \ensuremath\mathbf{x}_i=[F_1,F_2]^{\mathsf T}$"></SPAN> given in the table below?  Compute
the values of the functions <!-- MATH
 $f_k(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="41" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img113.gif"
 ALT="$ f_k(\ensuremath\mathbf{x}_i)$"></SPAN> for our model <!-- MATH
 $\ensuremath\boldsymbol{\Theta}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="14" ALIGN="BOTTOM" BORDER="0"
 SRC="img61.gif"
 ALT="$ \ensuremath\boldsymbol{\Theta}$"></SPAN> as the
right-hand side of eq.&nbsp;<A HREF="#eq:log-decision-rule">5</A>: <!-- MATH
 $f_k(\ensuremath\mathbf{x}_i) = \log
p(\ensuremath\mathbf{x}_i|q_k,\ensuremath\boldsymbol{\Theta}) + \log P(q_k|\ensuremath\boldsymbol{\Theta})$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="236" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img114.gif"
 ALT="$ f_k(\ensuremath\mathbf{x}_i) = \log
p(\ensuremath\mathbf{x}_i\vert q_k,\ensuremath\boldsymbol{\Theta}) + \log P(q_k\vert\ensuremath\boldsymbol{\Theta})$"></SPAN>, proportional to the log of the
posterior probability of <!-- MATH
 $\ensuremath\mathbf{x}_i$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img44.gif"
 ALT="$ \ensuremath\mathbf{x}_i$"></SPAN> belonging to class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN>.

<P>

<P>
<BR>

<DIV ALIGN="CENTER">

  <TABLE CELLPADDING=3 BORDER="1">
<TR><TD ALIGN="CENTER">i</TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $\ensuremath\mathbf{x}_i=[F_1,F_2]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="87" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img112.gif"
 ALT="$ \ensuremath\mathbf{x}_i=[F_1,F_2]^{\mathsf T}$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $f_{\text{/a/}}(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="52" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img115.gif"
 ALT="$ f_{\text{/a/}}(\ensuremath\mathbf{x}_i)$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $f_{\text{/e/}}(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="51" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img116.gif"
 ALT="$ f_{\text{/e/}}(\ensuremath\mathbf{x}_i)$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $f_{\text{/i/}}(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="49" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img117.gif"
 ALT="$ f_{\text{/i/}}(\ensuremath\mathbf{x}_i)$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $f_{\text{/o/}}(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="52" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img118.gif"
 ALT="$ f_{\text{/o/}}(\ensuremath\mathbf{x}_i)$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER"><SMALL CLASS="SMALL"><!-- MATH
 $f_{\text{/y/}}(\ensuremath\mathbf{x}_i)$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="52" HEIGHT="28" ALIGN="MIDDLE" BORDER="0"
 SRC="img119.gif"
 ALT="$ f_{\text{/y/}}(\ensuremath\mathbf{x}_i)$"></SPAN> </SMALL></TD>
<TD ALIGN="CENTER">Most prob. class <SPAN CLASS="MATH"><IMG
 WIDTH="16" HEIGHT="25" ALIGN="MIDDLE" BORDER="0"
 SRC="img91.gif"
 ALT="$ q_k$"></SPAN></TD>
</TR>
<TR><TD ALIGN="CENTER">1</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[400,1800]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img120.gif"
 ALT="$ [400,1800]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
<TR><TD ALIGN="CENTER">2</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[400,1000]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img121.gif"
 ALT="$ [400,1000]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
<TR><TD ALIGN="CENTER">3</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[530,1000]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img122.gif"
 ALT="$ [530,1000]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
<TR><TD ALIGN="CENTER">4</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[600,1300]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img123.gif"
 ALT="$ [600,1300]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
<TR><TD ALIGN="CENTER">5</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[670,1300]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img124.gif"
 ALT="$ [670,1300]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
<TR><TD ALIGN="CENTER">6</TD>
<TD ALIGN="CENTER"><!-- MATH
 $[420,2500]^{\mathsf T}$
 -->
<SPAN CLASS="MATH"><IMG
 WIDTH="74" HEIGHT="31" ALIGN="MIDDLE" BORDER="0"
 SRC="img125.gif"
 ALT="$ [420,2500]^{\mathsf T}$"></SPAN></TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
<TD ALIGN="CENTER">&nbsp;</TD>
</TR>
</TABLE>
</DIV>