sv_user.html

一种新颖的SVM算法

depend on the type of SV machine you have chosen, if you chose
regression estimation though, the following menu would appear :
<P>
<PRE>        General parameters
        ==================
         1. Bound on Alphas (C) 0
         2. Scaling             off
         3. Chunking            off
         4. Epsilon accuracy    0
         6. Number of Machines  1

         0. Exit

        Please enter your choice:</PRE>
<P>
This is identical to the options for the Pattern Recognition case,
except that Pattern Recognition does not have epsilon as a parameter,
so that option is not included.  For the moment, leave chunking off
(you only need chunking for large data sets i.e. those &gt; 700
examples), and leave the number of machines set to 1.  This leaves
alpha bounds, epsilon accuracy, and scaling.
<P>
The bound on the alphas is simply the value of C which you wish to
use.  A value of 0 will set C to infinity, which corresponds to
requiring linear separability in the pattern recognition case.
Another way of thinking of it is that the Value of C represents how
much you punish errors (i.e. examples that lie on the wrong side of
the hyperplane).  The larger value of C, the less errors you are
allowing the decision rule to have.
<P>
Epsilon accuracy is simply a floating point number, which controls the
size of the epsilon insensitive zone.
<P>
Scaling is used when the data in the input file is not properly
scaled.  Scaling can be done either globally (i.e. all values are
scaled the same) or locally (each individual attribute is scaled
independently).  As a guideline you may wish to think of it this way;
if the attributes are all of the same type (e.g. pixel values) then
scale globally, if they are of different types (e,g, age, height,
weight) then scale locally. When you select the scaling option, the
program first asks if you want to scale the data, then it asks if all
attributes are to be scaled with the same factor.  Answering Y
corresponds to global scaling and N corresponds to local scaling.  You
are then asked to specifiy the lower and upper bounds for the scaled
data e.g. -1 and 1, or 0 and 1.
<P>
<H2><A NAME="SECTION00024000000000000000">Setting Kernel Specific Paramters</A></H2>
<P>
The final option on the SVM paramter menu is used to set the kernel
specific paramters.  This is obviously greatly dependent on the kernel
you have chosen to use.  Some kernels such as Inifinite Dimensional
Splines do not have any paramters, most kernels do though.  As an
example, if you are using the 2 layer Neural Network kernel (see
section 2.2), then selecting option 4 would result in the SVM program
prompting you to enter the values for the kernel's two paramters - b
and c.
<P>
Once again, there is no rule for which values of paramters are good
for different kernels on different data sets.  You simply have to try
different values and try to develop an understanding of how the kernel
works.
<P>
<H2><A NAME="SECTION00025000000000000000">Setting the Expert Parameters</A></H2>
<P>
Don't.  There you have it in one word, unless you know exactly what
you want to achieve then its not a good idea to mess with these
settings.  There is (like everything else) one exception though.  You
can choose which optimiser to use.  As a suggestion, you may want to
use LOQO for pattern recognition, and MINOS for regression
estimation.  Do not use the third option BOTTOU as that optimiser has
not been fully implemented yet.
<P>
<H2><A NAME="SECTION00026000000000000000">Life After Parameter Setting</A></H2>
<P>
After you have chosen the parameters, you are now ready to try them.
If you reached the parameter menu by running the SVM program without
specifying a parameter file, then when you select exit the SV machine
will run on the data with the paramters you have just specified (and
these parameters will be saved in tmp.params).
If you are using paragen, then you will be given the option to save
the parameters (at which point you have to specify a filename),
display the parameters, or change them if you are not happy with your
selection (You will also have an option ``special commands'' which
allows you to add parameter sets, but its best to ignore this for the
time being).  You can now run the SV machine with your parameter file
specified on the command line, and see what results you get.
<P>
<H1><A NAME="SECTION00030000000000000000">Output from the SVM Program</A></H1>
<P>
Generating parameter files and running the SVM program on various sets
of data may be fun, but it isn't of much practical use unless you can
understand some of the output which the support vector machine gives.
<P>
So here's some example output (for pattern recognition), and we'll go
through it step by step :
<P>
<PRE>rm: fort.9: No such file or directory
Fri Jan  2 11:27:28 WET 1998

        SV Machine parameters
        =====================
        Pattern Recognition

        Full polynomial

        Alphas unbounded
        Input values will be globally scaled between 0 and 1.
        Training data will not be posh chunked.
        Training data will not be sporty chunked.
        Number of SVM: 1

        Degree of polynomial: 2.
        Kernel Scale Factor: 256.
        Kernel Threshold: 1.
----------------------------------------
----------------------------------------
Positive SVs: 12 13 16 41 111 114 157 161 
Negative SVs: 8 36 126 138 155 165 
There are 14 SVs (8 positive and 6 negative).
Max alpha size: 3.78932
B0 is -1.87909
Objective = 9.71091
Fri Jan  2 11:27:33 WET 1998
Training set: 
Total samples:          200
Positive samples:       100
of which errors:        0
Negative samples:       100
of which errors:        0

----------------------------------------
Test set: 
Total samples:          50
Positive samples:       25
of which errors:        0
Negative samples:       25
of which errors:        0

There are 1 lagrangian multipliers per support vector.
No.     alpha(0)        Deviation
    8      1.86799      3.77646e-08
   12     0.057789      6.97745e-08
   13      2.75386                  
   16     0.889041      -1.63897e-08
   36      1.53568      5.93671e-08
   41     0.730079      3.91323e-09
  111     0.359107      -1.38041e-07
  114      0.88427      -9.06655e-08
  126     0.561356      1.15792e-07
  138        2.558      2.0497e-08
  155      2.50095      -1.24475e-08
  157     0.247452      -1.35147e-07
  161      3.78932      -1.2767e-07
  165     0.686947      1.03066e-07
Finished checking support vector accuracy.
Total   deviation is 9.30535e-07      No. of SVs: 14
Average deviation is 6.64668e-08
Minimum alpha     is 0.057789
Maximum alpha     is 3.78932</PRE>
<P>
The SVM program uses a different type of optimiser to construct the
rule, depending on which one you selected when setting the parameters.
When using LOQO the optimiser gives no output, but MINOSS, the other
optimiser, gives an output of the following form :
<P>
<PRE> ==============================
 M I N O S    5.4    (Dec 1992)
 ==============================

 Begin SV_TEST         hi!
OPTIMAL SOLUTION FOUND (0)
----------------------------------------</PRE>
<P>
In this case, the optimiser signals that an optimal solution was
found.  If the data is scaled badly, or the data is inseperable (and
the bound on the alphas is infinite), then an error may occur here.
Therefore, you will have to ensure the scaling options are set
correctly, and you may have to change the bound on the alpha values
(the value of C).
<P>
The next section informs the user how many support vectors there are,
and lists the example numbers of those examples which were support
vectors.  This section also indicates the largest alpha value, and the
value of b0.
<P>
Next is important information as to how the SVM
performed on both the training set and the test set.  In the case of
pattern recognition (as shown above), the ouput indicates the number
of positive and negative samples, and the number of those which were
misclassified in both the training and the test set.  For instance, in
the example above, all of the examples in the training set were 
classified correctly.  Obviously, in regression estimation, you are
not interested in classification.  When running the SVM program to
perform regression estimation, various measures of error are displayed
here.  The user is given the average (absolute) error on the training
set.  Also, the totals and averages are displayed for both absolute
and squared error on the training set.
<P>
Following the performance statistics, a list of the values of the
alphas (lagrange multipliers) for each support vector is given, along
with its deviation (how far away the support vector is from the
boundary of the margin). Finally some statistics are given, indicating
the minimum and maximum alpha values.
<P>
<H1><A NAME="SECTION00040000000000000000">Further Information</A></H1>
<P>
There is an on-line version of the support vector machine which has
been developed in the department.  The web site has a graphical
interface which allows you to plot a few points and see what decision
boundary is produced.  Be warned that when the site has many users, it
can be a little slow.  It is sometimes useful to see what the SVM
program actually does though (at least in two dimensions), and the
page also provides links to other SVM sites.  
The web address of the page is :
<P>
http://svm.cs.rhbnc.ac.uk
<P>
<H1><A NAME="SECTION00050000000000000000">  About this document ... </A></H1>
<P>
 <STRONG>Support Vector Machine - User Manual</STRONG><P>
This document was generated using the <A HREF="http://www-dsed.llnl.gov/files/programs/unix/latex2html/manual/"><STRONG>LaTeX</STRONG>2<tt>HTML</tt></A> translator Version 96.1-h (September 30, 1996) Copyright &#169; 1993, 1994, 1995, 1996,  <A HREF="http://cbl.leeds.ac.uk/nikos/personal.html">Nikos Drakos</A>, Computer Based Learning Unit, University of Leeds. <P> The command line arguments were: <BR>
<STRONG>latex2html</STRONG> <tt>-split 0 -html_version 3.0 sv_user</tt>. <P>The translation was initiated by Mark O. Stitson on Wed Jun  3 14:45:37 BST 1998<HR><IMG WIDTH=37 HEIGHT=24 ALIGN=BOTTOM ALT="next" SRC="file:/usr/lib/latex2html/icons/next_motif.gif"> <IMG WIDTH=26 HEIGHT=24 ALIGN=BOTTOM ALT="up" SRC="file:/usr/lib/latex2html/icons/up_motif_gr.gif"> <IMG WIDTH=63 HEIGHT=24 ALIGN=BOTTOM ALT="previous" SRC="file:/usr/lib/latex2html/icons/previous_motif.gif">   <BR>
<P><ADDRESS>
<I>Mark O. Stitson <BR>
Wed Jun  3 14:45:37 BST 1998</I>
</ADDRESS>
</BODY>
</HTML>