mysvm.htm

介绍支持向量机SVM介绍的参考文献以及程序源代码

        deviation</TD></TR></TBODY></TABLE>
      <H4>Optimizer parameters:</H4>
      <TABLE border=1>
        <TBODY>
        <TR>
          <TD>working_set_size <EM>int</EM></TD>
          <TD>optimize this much examples in each iteration (default: 
10)</TD></TR>
        <TR>
          <TD>max_iterations <EM>int</EM></TD>
          <TD>stop after this much iterations</TD></TR>
        <TR>
          <TD>shrink_const <EM>int</EM></TD>
          <TD>fix a variable to the bound if it is optimal for this much 
            iterations</TD></TR>
        <TR>
          <TD>is_zero <EM>float</EM></TD>
          <TD>numerical precision (default: 1e-10)</TD></TR>
        <TR>
          <TD>descend <EM>float</EM></TD>
          <TD>make this much descend on the target function in each 
          iteration</TD></TR>
        <TR>
          <TD>convergence_epsilon <EM>float</EM></TD>
          <TD>precision on the KKT conditions (default: 1e-3 for pattern 
            recognition and 1e-4 for regression)</TD></TR>
        <TR>
          <TD>kernel_cache <EM>int</EM></TD>
          <TD>size of the cache for kernel evaluations im MB (default: 
        40)</TD></TR></TBODY></TABLE>
      <H4>Training algorithms</H4>
      <TABLE border=1>
        <TBODY>
        <TR>
          <TD>cross_validation <EM>int</EM></TD>
          <TD>do cross validation on the training examples with the given 
            number of chunks</TD></TR>
        <TR>
          <TD>cv_inorder</TD>
          <TD>do cross validation in the order the examples are given in</TD></TR>
        <TR>
          <TD>cv_window <EM>int</EM></TD>
          <TD>do cross validation by moving a window of the given number of 
            chunks over the training data. (Implies cv_inorder)</TD></TR>
        <TR>
          <TD>search_C <EM>[am]</EM></TD>
          <TD>find an optimal C in the range of cmin to cmax by 
            <STRONG>A</STRONG>dding or <STRONG>M</STRONG>ultiplying the current 
            C by cdelta</TD></TR>
        <TR>
          <TD>cmin</TD>
          <TD>lower bound for search_C</TD></TR>
        <TR>
          <TD>cmax</TD>
          <TD>upper bound for search_C</TD></TR>
        <TR>
          <TD>cdelta</TD>
          <TD>step size for search_C</TD></TR></TBODY></TABLE><A name=kerneldef>
      <H3>Kernel definition </H3></A>The kernel definition lets you choose the 
      type of kernel function to use and its parameters. It starts with the line 
      <TT>@kernel</TT>
      <P>
      <TABLE border=1>
        <TBODY>
        <TR>
          <TH>name</TH>
          <TH>kernel type</TH>
          <TH>parameters</TH></TR>
        <TR>
          <TD>dot</TD>
          <TD>inner product</TD>
          <TD>none</TD></TR>
        <TR>
          <TD>polynomial</TD>
          <TD>polynomial (x*y+1)^d</TD>
          <TD>degree <EM>int</EM></TD></TR>
        <TR>
          <TD>radial</TD>
          <TD>radial basis function exp(-gamma ||x-y||^2)</TD>
          <TD>gamma <EM>float</EM></TD></TR>
        <TR>
          <TD>neural</TD>
          <TD>two layered neural net tanh(a x*y+b)</TD>
          <TD>a <EM>float</EM>, b <EM>float</EM></TD></TR>
        <TR>
          <TD>anova</TD>
          <TD>(RBF) anova kernel</TD>
          <TD>gamma <EM>float&gt;/em&gt;, degree <EM>int</EM></EM></TD></TR>
        <TR>
          <TD>user</TD>
          <TD>user definable kernel</TD>
          <TD>param_i_1 ... param_i_5 <EM>int</EM>, param_f_1 ... param_f_5 
            <EM>float</EM></TD></TR>
        <TR>
          <TD>user2</TD>
          <TD>user definable kernel 2</TD>
          <TD>param_i, param_f</TD></TR>
        <TR>
          <TD>sum_aggregation</TD>
          <TD>sum of other kernels</TD>
          <TD>number_parts <EM>int</EM>, range <EM>int</EM> <EM>int</EM>, 
            followed by <TT>number_parts</TT> kernel definitions</TD></TR>
        <TR>
          <TD>prod_aggregation</TD>
          <TD>product of other kernels</TD>
          <TD>number_parts <EM>int</EM>, range <EM>int</EM> <EM>int</EM>, 
            followed by <TT>number_parts</TT> kernel 
      definitions</TD></TR></TBODY></TABLE><A name=exampledef>
      <H3>Example sets </H3></A>An example set consists of the learning 
      attributes for each example, its classification (or functional value) and 
      its lagrangian multiplier (actually, you don't need to supply the 
      lagrangian multiplier for training and you don't even have to supply the 
      functional value for prediction. But you could). The examples can be given 
      in two different formats: dense and sparse. Note that you can change the 
      data format 
      <P>The examples set definition starts with <TT>@examples</TT>. Note that 
      each example has to be in an own line. 
      <P>WARNING: Giving real number you can also use a colon instead of a 
      decimal dot ("<TT>1234,56</TT>" instead of "<TT>1234.56</TT>", german 
      style). Therefore something like "<TT>1,234.56</TT>" does not work! 
      <H4>common parameters:</H4>
      <TABLE border=1>
        <TBODY>
        <TR>
          <TD>format <EM>F</EM></TD>
          <TD>Format of examples where <EM>F</EM> is either "sparse" or a 
            string containing "x", "y" or "a". The format strings define the 
            position of the attributes <EM>x</EM>, the funtional value 
            <EM>y</EM> and the lagrangian multiplier <EM>a</EM> in an example. 
            "x" has to be set. The default format is "yx", but you can set 
            another default in the parameters definition.</TD></TR>
        <TR>
          <TD>dimension <EM>int</EM></TD>
          <TD>number of attributes. If the dimension is not given it is set 
            from the examples (maximum dimension in sparse format, dimension 
            from the first line in dense format).</TD></TR>
        <TR>
          <TD>number <EM>int</EM></TD>
          <TD>total number of examples. A warning is issued when a wrong 
            number of examples is given</TD></TR>
        <TR>
          <TD>b <EM>float</EM></TD>
          <TD>additional constant of the hyperplane</TD></TR>
        <TR>
          <TD>delimiter <EM>char</EM></TD>
          <TD>character by which the attributes of an example are separated 
            (default: space). You can set a default in the parameters section. 
            Be careful if you set the delimiter to "," or 
      "."!</TD></TR></TBODY></TABLE>
      <H4>sparse format:</H4>In the sparse data format, only non-zero attributes 
      have to be given. For each non-zero attribute you give its attribute 
      number (starting at 1) and its value, separated by a colon. The functional 
      value is given by y:<EM>float</EM> (the "y:" is optional here!) and the 
      lagrangian multiplier by a:<EM>float</EM>. 
      <P>Example: The following lines all define the same example: 
      <UL>
        <LI><SAMP>1:-1 2:0 3:1.2 y:2 a:0</SAMP> 
        <LI><SAMP>3:1.2 y:2 1:-1</SAMP> 
        <LI><SAMP>3:1.2 2 1:-1</SAMP> </LI></UL>
      <H4>dense format</H4>The dense format consists of all attributes and (if 
      defined so) the functional values and the lagrangian multipliers listed in 
      the order given by the <TT>format</TT> parameter. 
      <P>Example: The following lines all define the same example as above: 
      <UL>
        <LI>With "<SAMP>format yx</SAMP>" (default) : "<SAMP>2 -1 0 1.2</SAMP>" 
        <LI>With "<SAMP>format xya</SAMP>" it is "<SAMP>-1 0 1.2 2 0</SAMP>" 
        <LI>And with "<SAMP>format xy</SAMP>" and "<SAMP>delimiter ','</SAMP>" 
        the example reads "<SAMP>-1,,1.2,2</SAMP>" </LI></UL>
      <H2>References </H2><!-- literatur bibkey Joachims/99a -or bibkey Vapnik/98a -or bibkey Schoelkopf/etal/2000a -or bibkey schoelkopf/etal/99a -->
      <TABLE>
        <TBODY>
        <TR>
          <TD vAlign=top><A 
            name=Schoelkopf/etal/2000a>[Schoelkopf/etal/2000a]</A></TD>
          <TD vAlign=top>Sch&ouml;lkopf, Bernhard and Smola, Alex J. and 
            Williamson, Robert C. and Bartlett, Peter L., <I>New Support Vector 
            Algorithms</I>. Neural Computation, Vol. 12, 2000.<BR></TD></TR>
        <TR>
          <TD vAlign=top><A name=Joachims/99a>[Joachims/99a]</A></TD>
          <TD vAlign=top>T. Joachims, 11 in: <I>Making large-Scale SVM 
            Learning Practical</I>. Advances in Kernel Methods - Support Vector 
            Learning, B. Sch&ouml;lkopf and C. Burges and A. Smola (ed.), MIT Press, 
            1999.<BR><A 
            href="http://www-ai.cs.uni-dortmund.de/DOKUMENTE/joachims_99a.ps.gz">Online 
            [Postscript (gz)]</A> &nbsp;<A 
            href="http://www-ai.cs.uni-dortmund.de/DOKUMENTE/joachims_99a.pdf">[PDF]</A> 
            &nbsp; </TD></TR>
        <TR>
          <TD vAlign=top><A 
          name=Schoelkopf/etal/99a>[Schoelkopf/etal/99a]</A></TD>
          <TD vAlign=top>Sch&ouml;lkopf, Bernhard and Williamson, Robert C. and 
            Smola, Alex J. and Shawe-Taylor, John, <I>SV Estimation of a 
            Distribution's Support</I>. Neural Information Processing Systems 
            12, Solla, S.A. and Leen, T.K. and Müller, K.-R. (ed.), MIT Press, 
            2000, forthcoming.<BR></TD></TR>
        <TR>
          <TD vAlign=top><A name=Vapnik/98a>[Vapnik/98a]</A></TD>
          <TD vAlign=top>V. Vapnik, <I>Statistical Learning Theory</I>. Wiley, 
            1998.<BR></TD></TR></TBODY></TABLE><!-- /literatur --><!-- footer --></TD>
    <TD>&nbsp;&nbsp;</TD></TR>
  <TR>
    <TD colSpan=3>&nbsp;</TD></TR></TBODY></TABLE>
<TABLE border=0 cellPadding=0 cellSpacing=0 width="100%">
  <TBODY>
  <TR>
    <TD bgColor=#99cdff height=25 noWrap vAlign=bottom><A 
      href="http://www-ai.cs.uni-dortmund.de/index.eng.html" 
      onmousedown="imgClick('img_bot_ls8news'); return true" 
      onmouseout="imgNormal('img_bot_ls8news'); return true" 
      onmouseover="imgOver('img_bot_ls8news'); return true"><IMG alt="LS8 News" 
      border=0 height=25 name=img_bot_ls8news src="mySVM.files/ls8news.gif" 
      width=77></A> &nbsp; <A 
      href="http://www-ai.cs.uni-dortmund.de/FORSCHUNG/index.eng.html" 
      onmousedown="imgClick('img_bot_forschung'); return true" 
      onmouseout="imgNormal('img_bot_forschung'); return true" 
      onmouseover="imgOver('img_bot_forschung'); return true"><IMG alt=Research 
      border=0 height=25 name=img_bot_forschung 
      src="mySVM.files/forschung.eng.gif" width=82></A><A 
      href="http://www-ai.cs.uni-dortmund.de/SOFTWARE/index.eng.html" 
      onmousedown="imgClick('img_bot_software'); return true" 
      onmouseout="imgNormal('img_bot_software'); return true" 
      onmouseover="imgOver('img_bot_software'); return true"><IMG alt=Software 
      border=0 height=25 name=img_bot_software src="mySVM.files/software.gif" 
      width=83></A><A 
      href="http://www-ai.cs.uni-dortmund.de/PARTNER/index.eng.html" 
      onmousedown="imgClick('img_bot_partner'); return true" 
      onmouseout="imgNormal('img_bot_partner'); return true" 
      onmouseover="imgOver('img_bot_partner'); return true"><IMG alt=Partner 
      border=0 height=25 name=img_bot_partner src="mySVM.files/partner.gif" 
      width=73></A> &nbsp; <A 
      href="http://www-ai.cs.uni-dortmund.de/LEHRE/lehre.html" 
      onmousedown="imgClick('img_bot_lehre'); return true" 
      onmouseout="imgNormal('img_bot_lehre'); return true" 
      onmouseover="imgOver('img_bot_lehre'); return true"><IMG alt=Teaching 
      border=0 height=25 name=img_bot_lehre src="mySVM.files/lehre.eng.gif" 
      width=79></A> &nbsp; <A 
      href="http://www-ai.cs.uni-dortmund.de/PERSONAL/personal.eng.html" 
      onmousedown="imgClick('img_bot_personal'); return true" 
      onmouseout="imgNormal('img_bot_personal'); return true" 
      onmouseover="imgOver('img_bot_personal'); return true"><IMG alt=Staff 
      border=0 height=25 name=img_bot_personal 
      src="mySVM.files/personal.eng.gif" width=54></A><A 
      href="http://www-ai.cs.uni-dortmund.de/UNIVERSELL/index.eng.html" 
      onmousedown="imgClick('img_bot_allgemein'); return true" 
      onmouseout="imgNormal('img_bot_allgemein'); return true" 
      onmouseover="imgOver('img_bot_allgemein'); return true"><IMG alt=General 
      border=0 height=25 name=img_bot_allgemein 
      src="mySVM.files/allgemein.eng.gif" width=74></A><A 
      href="http://www-ai.cs.uni-dortmund.de/INTERN/intern.html" 
      onmousedown="imgClick('img_bot_intern'); return true" 
      onmouseout="imgNormal('img_bot_intern'); return true" 
      onmouseover="imgOver('img_bot_intern'); return true"><IMG alt=Internal 
      border=0 height=25 name=img_bot_intern src="mySVM.files/intern.eng.gif" 
      width=85></A></TD>
    <TD align=middle bgColor=#99cdff><SMALL>Hits:</SMALL> <IMG 
      src="mySVM.files/server-cntr.gif"></TD>
    <TD align=right bgColor=#99cdff height=25 noWrap vAlign=bottom><A 
      href="http://www-ai.cs.uni-dortmund.de/Harvest/brokers/www-ai/query.eng.html" 
      onmousedown="imgClick('img_bot_search'); return true" 
      onmouseout="imgNormal('img_bot_search'); return true" 
      onmouseover="imgOver('img_bot_search'); return true"><IMG alt=Search 
      border=0 height=25 name=img_bot_search src="mySVM.files/search.gif" 
      width=30></A><A href="mailto:[e-mail]" 
      onmousedown="imgClick('img_bot_mail'); return true" 
      onmouseout="imgNormal('img_bot_mail'); return true" 
      onmouseover="imgOver('img_bot_mail'); return true"><IMG 
      alt="Send email to [e-mail]" border=0 height=25 
      name=img_bot_mail src="mySVM.files/mail.gif" width=38></A><IMG 
      alt="no german" border=0 height=25 src="mySVM.files/no_deutsch.gif" 
      width=32></TD></TR></TBODY></TABLE><!-- /footer --></BODY></HTML>