📄 reference.html
字号:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 3.0//EN"><!--Converted with LaTeX2HTML 96.1-h (September 30, 1996) by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds --><HTML><HEAD><TITLE>Support Vector Machine Reference Manual</TITLE><META NAME="description" CONTENT="Support Vector Machine Reference Manual"><META NAME="keywords" CONTENT="reference"><META NAME="resource-type" CONTENT="document"><META NAME="distribution" CONTENT="global"><LINK REL=STYLESHEET HREF="reference.css"></HEAD><BODY LANG="EN" > <IMG WIDTH=37 HEIGHT=24 ALIGN=BOTTOM ALT="next" SRC="file:/usr/lib/latex2html/icons/next_motif.gif"> <IMG WIDTH=26 HEIGHT=24 ALIGN=BOTTOM ALT="up" SRC="file:/usr/lib/latex2html/icons/up_motif_gr.gif"> <IMG WIDTH=63 HEIGHT=24 ALIGN=BOTTOM ALT="previous" SRC="file:/usr/lib/latex2html/icons/previous_motif.gif"> <BR><BR> <P> <H1 ALIGN=CENTER>Support Vector Machine <BR> Reference Manual</H1><P ALIGN=CENTER><STRONG>C. Saunders, M. O. Stitson, J. Weston<BR> Department of Computer Science<BR> Royal Holloway<BR> University of London<BR> <EM>e-mail:</EM> {C.Saunders,M.Stitson,J.Weston}@dcs.rhbnc.ac.uk - L. Bottou<BR> AT&T Speech and Image Processing Services Research Lab<BR> <EM>e-mail:</EM> leonb@research.att.com - B. Schölkopf, A. Smola<BR> GMD FIRST<BR> <EM>e-mail:</EM> {bs,smola}@first.gmd.de</STRONG></P><P><P ALIGN=CENTER><STRONG>July 10, 1998</STRONG></P><P><P>The Support Vector Machine (SVM) is a new type of learningmachine. The SVM is a general architecture that can be applied topattern recognition, regression estimation and other problems. Thefollowing researchers were involved in the development of theSVM:<P><TABLE COLS=3><COL ALIGN=LEFT><COL ALIGN=LEFT><COL ALIGN=LEFT><TR><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>A. Gammerman (RHUL)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>V. Vapnik (AT&T, RHUL)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>Y. LeCun (AT&T)</TD></TR><TR><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP> N. Bozanic (RHUL)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>L. Bottou (AT&T)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>C. Saunders (RHUL)</TD></TR><TR><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP> B. Schölkopf (GMD)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>A. Smola (GMD)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>M. O. Stitson (RHUL)</TD></TR><TR><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP> V. Vovk (RHUL)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>C. Watkins (RHUL)</TD><TD VALIGN=BASELINE ALIGN=LEFT NOWRAP>J. A. E. Weston (RHUL)</TD></TR></TABLE><P>The major reference is V.Vapnik, ``The Nature of Statistical LearningTheory'', Springer 1995.<P><H1><A NAME="SECTION00010000000000000000">Getting Started</A></H1><P>The Support Vector Machine (SVM) program allows a user to carry out pattern recognition and regression estimation, usingsupport vector techniques on some given data.<P>If you have any questions not answered by the documentation, you can e-mail us at:<P><TT>svmmanager@dcs.rhbnc.ac.uk</TT><P><H1><A NAME="SECTION00020000000000000000">Programs</A></H1><P>Release 1.0 of the RHUL SV Machine comes with a set of seven programs(<TT>sv, paragen, loadsv, transform_sv, snsv, ascii2bin, bin2ascii</TT>).<P><UL><LI> <TT>sv</TT> - the main SVM program<LI> <TT>paragen</TT> - program for generating parameter sets for the SVM<LI> <TT>loadsv</TT> - load a saved SVM and classify a new data set<LI> <TT>transform_sv</TT> - special SVM program for image recognition, that implements virtual support vectors [<A HREF="reference.html#bernhard">BS97</A>].<LI> <TT>snsv</TT> - program to convert SN format to our format<LI> <TT>ascii2bin</TT> - program to convert our ASCII format to our binaryformat<LI> <TT>bin2ascii</TT> - program to convert our binary format to our ASCIIformat</UL><P>The rest of this document will describe these programs. To find outmore about SVMs, see the bibliography. We will not describe how SVMswork here.<P>The first program we will describe is the <TT>paragen</TT> program, as itspecifies all parameters needed for the SVM.<P><H1><A NAME="SECTION00030000000000000000"><TT>paragen</TT></A></H1><A NAME="paragen"> </A><P>When using the support vector machine for any given task, it is alwaysnecessary to specify a set of parameters. These parameters includeinformation such as whether you are interested in pattern recognitionor regression estimation, what kernel you are using, what scaling isto be done on the data, etc... <TT>paragen</TT> generates parameter files used by the SVM program, if no filewas generated the user will be asked interactively.<P><TT>paragen</TT> is run by the following command line :<P><TT>paragen [ <load parameter file> [<save parameter file>] ]</TT><P>The parameter file is optional, and obviously cannot be included thefirst time you run <TT>paragen</TT> as you have not created a parameter filebefore. If however, you have a parameter file which is similar tothe one you want to use, by specifying that file as part of thecommand line the program will start with all of the parameters set tothe relevant values, allowing you to make a couple of changes andthen save the file under a different name. The second option is tospecify the name of the file you want to save the parameters to. Thiscan also be done by selecting a menu option within <TT>paragen</TT>. If no saveparameter file argument is given it is assumed that you wish to saveover the same file name as given in the load parameter file argument.<P><H2><A NAME="SECTION00031000000000000000">Traversing the menu system</A></H2><P><TT>paragen</TT> uses a simple text basedmenu system. The menus are organized in a tree structure which can be traversed by typing the number of the desiredoption, followed by return. Option 0 (labelled ``Exit'')is in each menu at each branch of the tree. Choosing option 0 always traverses up one level of the tree. If you are already atthe top of the tree, it exits the program.<P><H2><A NAME="SECTION00032000000000000000">The top level menu</A></H2><P>The first menu givesyou the option of displaying, entering, or saving parameters.When the menu appears, if youchoose the enter parameters option, this process is identical tospecifying parameters interactively when running the SVM. The menu looks like this:<P><PRE>SV Machine Parameters===================== 1. Enter parameters 2. Load parameters 3. Save parameters (pattern_test) 4. Save parameters as... 5. Show parameters 0. Exit</PRE><P>Options 2,3 and 4 are straight forward. Option 5 displays the chosen parameters and option 1 allows the parameters to be entered.We now describe the branches of the menu tree after choosingoption 1 (``Enter parameters'') in detail.<P><H2><A NAME="SECTION00033000000000000000">Enter Parameters</A></H2><P>If you choose to enter parameters,then you are faced with a list of the current parameter settings, and amenu. An example of which is shown below :<P><PRE> SV Machine parameters ===================== No kernel specified Alphas unbounded Input values will not be scaled. Training data will not be posh chunked. Training data will not be sporty chunked. Number of parameter sets: 1 Optimizer: 3 SV zero threshold: 1e-16 Margin threshold: 0.1 Objective zero tolerance: 1e-07 1. Set the SV Machine type 2. Set the Kernel type 3. Set general parameters 4. Set kernel specific parameters 5. Set expert parameters 0. Exit Please enter your choice:</PRE><P>Each of these menu options allow the users to specify differentaspects of the Support Vector Machine that they wish to use, and eachone will now be dealt with in turn.<P><H2><A NAME="SECTION00034000000000000000">Setting the SV Machine Type</A></H2><P>When option 1 is chosen, the following menu appears:<P><PRE> Type of SV machine ================== 1 Pattern Recognition 2 Regression Estimation 6 Multiclass Pattern Recognition Please enter the machine type: (0)</PRE><P>After entering 1, 2 or 6 at the prompt (and pressing return), you are giventhe top-level parameter menu again. If you look at the line below``SV Machine Parameters'', the type of SV machine which was selectedshould be displayed.<P>After selecting the SV machine type, the user must decide which kernelto use.<P><H2><A NAME="SECTION00035000000000000000">Setting the Kernel Type</A></H2><P>Option 2 from the SV machine parameters menu allows the kernel type to be chosen. For a detaileddescription of the kernel types see the appendix.The menu will look like this:<P><PRE> Type of Kernel ============== 1 Simple Dot Product 2 Vapnik's Polynomial 3 Vovk's Polynomial 4 Vovk's Infinite Polynomial 5 Radial Basis Function 6 Two Layer Neural Network 7 Infinite dimensional linear splines 8 Full Polynomial (with scaling) 9 Weak-mode Regularized Fourier 10 Semi-Local (polynomial & radial basis) 11 Strong-mode Regularized Fourier 17 Anova 1 18 Generic Kernel 1 19 Generic Kernel 2</PRE><P>Many of the kernel functions have one or more free parameters,the values of which can be set using option 4 ``Set kernelspecific parameters'' in the SV Machine parameters menu (one branch ofthe tree up from this menu). For example, using the polynomial kernel (2) ``Vapnik's polynomial'' one can control the free parameter <I>d</I>, the degree of polynomial.<P><H3><A NAME="SECTION00035100000000000000">Implementing new kernel functions</A></H3><P>Options 18 and 19 are special convenience kernel functions that have beenincluded for experienced users who wish to implement their own kernel functions.<P>Kernel functions are written as a C++ class that inherits mostof its functionality from a base class. When you wish to add a new kernel,instead of adding a new class and having to change various interface routines and the Makefiles you can simply change the function <TT>kernel_generic_1_c::calcK(...)</TT> or <TT>kernel_generic_2_c::calcK(...)</TT> and choose to use this kernelfrom the menu options after re-compilation.<P>Both generic kernels havefive (potential) free parameters labelled <TT>a_val</TT>, <TT>b_val</TT> and so on which can be set in the usual way in the ``Set kernelspecific parameters'' menu option.<P><H2><A NAME="SECTION00036000000000000000">Setting the General Parameters</A></H2><P>Option 3 of the SVM parameter menu allows the user to set the free parameters for the SVM (in the pattern recognition case, the size ofthe upper bound on the Lagrangian variables, i.e. the box constraints,<I>C</I>, and in regression estimation, <I>C</I> and the choice of <IMG WIDTH=6 HEIGHT=7 ALIGN=BOTTOM ALT="tex2html_wrap_inline518" SRC="img1.gif"> for the the <IMG WIDTH=6 HEIGHT=7 ALIGN=BOTTOM ALT="tex2html_wrap_inline518" SRC="img1.gif">-insensitive loss function.) Various other miscellaneousoptions have also been grouped together here: scaling strategy, chunking strategy and multi-class pattern recognition strategy.This menu has different options depending on the type of SV Machine chosen:pattern recognition, regression estimation or multi-class pattern recognition.<P>For the pattern recognition SVM the following options are given:<P><PRE> General parameters ================== 1. Bound on Alphas (C) 0 2. Scaling off 3. Chunking 0. Exit</PRE><P>For the regression SVM the following options are given:<P><PRE> General parameters ================== 1. Bound on Alphas (C) 0 2. Scaling off 3. Chunking 4. Epsilon accuracy 0 0. Exit</PRE><P>For the multi-class SVM the following options are given:<P><PRE> General parameters ================== 1. Bound on Alphas (C) 0 2. Scaling off 3. Chunking off 7. Multi-class Method 1 8. Multi-class Continuous Classes 0⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -