📄 svm_light.htm
字号:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<!-- saved from url=(0046)http://www.cs.cornell.edu/People/tj/svm_light/ -->
<HTML><HEAD><TITLE>SVM-Light Support Vector Machine</TITLE>
<META http-equiv=Content-Type content="text/html; charset=windows-1252">
<META content="MSHTML 6.00.6000.16674" name=GENERATOR>
<META content=8.0.3514 name=Version>
<META content=11/26/96 name=Date>
<META content="C:\Programme\Microsoft Office\Office\HTML.DOT"
name=Template></HEAD>
<BODY vLink=#800080 link=#0000ff bgColor=#ffffff>
<TABLE cellSpacing=0 cellPadding=5 border=0>
<TBODY>
<TR>
<TD vAlign=top width="14%">
<H2><A href="http://www-ai.cs.uni-dortmund.de/" target=_top><IMG height=81
src="svm_light_files/eier_graybg.gif" width=100 border=0></A></H2></TD>
<TD vAlign=top width="75%">
<H1 align=center>SVM<I><SUP>light</SUP> </H1></I>
<H1 align=center>Support Vector Machine</H1>
<P align=center>Author: <A href="http://www.joachims.org/"
target=_top>Thorsten Joachims</A> <<A
href="mailto:thorsten@joachims.org">thorsten@joachims.org</A>> <BR><A
href="http://www.cornell.edu/" target=_top>Cornell University</A> <BR><A
href="http://www.cs.cornell.edu/" target=_top>Department of Computer
Science</A> </P>
<P align=center>Developed at: <BR><A href="http://www.uni-dortmund.de/"
target=_top>University of Dortmund</A>, <A
href="http://www.informatik.uni-dortmund.de/" target=_top>Informatik</A>,
<A href="http://www-ai.informatik.uni-dortmund.de/"
target=_top>AI-Unit</A> <BR><A href="http://www.sfb475.uni-dortmund.de/"
target=_top>Collaborative Research Center on 'Complexity Reduction in
Multivariate Data' (SFB475)</A> </P>
<P align=center>Version: 6.01 <BR>Date: 02.09.2004</P></TD>
<TD vAlign=top width="11%">
<H2><IMG height=80 src="svm_light_files/culogo_125.gif"
width=80></H2></TD></TR></TBODY></TABLE>
<H2>Overview</H2>
<P>SVM<I><SUP>light</I></SUP> is an implementation of Support Vector Machines
(SVMs) in C. The main features of the program are the following: </P>
<UL>
<LI>fast optimization algorithm
<UL>
<LI>working set selection based on steepest feasible descent
<LI>"shrinking" heuristic
<LI>caching of kernel evaluations
<LI>use of folding in the linear case </LI></UL>
<LI>solves classification and regression problems. For multivariate and
structured outputs use <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_struct.html">SVM<I><SUP>struct</I></SUP></A>.
<LI>solves ranking problems (e. g. learning retrieval functions in <A
href="http://striver.joachims.org/"><I>STRIVER</I></A> search engine).
<LI>computes XiAlpha-estimates of the error rate, the precision, and the
recall
<LI>efficiently computes Leave-One-Out estimates of the error rate, the
precision, and the recall
<LI>includes algorithm for approximately training large transductive SVMs
(TSVMs) (see also <A href="http://sgt.joachims.org/">Spectral Graph
Transducer</A>)
<LI>can train SVMs with cost models and example dependent costs
<LI>allows restarts from specified vector of dual variables
<LI>handles many thousands of support vectors
<LI>handles several hundred-thousands of training examples
<LI>supports standard kernel functions and lets you define your own
<LI>uses sparse vector representation </LI></UL>
<P><IMG height=16 src="svm_light_files/new.gif" width=32 border=0> <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_struct.html">SVM<I><SUP>struct</I></SUP></A>:
SVM learning for multivariate and structured outputs like trees, sequences, and
sets (available <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_struct.html">here</A>).</P>
<P><IMG height=16 src="svm_light_files/new.gif" width=32 border=0> <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_perf.html">SVM<SUP><I>perf</I></SUP></A>:
New training algorithm for linear classification SVMs that can be much faster
than SVM<SUP><I>light</I></SUP> for large datasets. It also lets you direcly
optimize multivariate performance measures like F1-Score, ROC-Area, and the
Precision/Recall Break-Even Point. (available <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_perf.html">here</A>).</P>
<H2>Description</H2>
<P>SVM<I><SUP>light</I></SUP> is an implementation of Vapnik's Support Vector
Machine [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Vapnik,
1995</A>] for the problem of pattern recognition, for the problem of regression,
and for the problem of learning a ranking function. The optimization algorithms
used in SVM<I><SUP>light</I></SUP> are described in [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
2002a</A> ]. [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
1999a</A>]. The algorithm has scalable memory requirements and can handle
problems with many thousands of support vectors efficiently. </P>
<P>The software also provides methods for assessing the generalization
performance efficiently. It includes two efficient estimation methods for both
error rate and precision/recall. XiAlpha-estimates [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
2002a</A>, <A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
2000b</A>] can be computed at essentially no computational expense, but they are
conservatively biased. Almost unbiased estimates provides leave-one-out testing.
SVM<I><SUP>light</I></SUP> exploits that the results of most leave-one-outs
(often more than 99%) are predetermined and need not be computed [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
2002a</A>].</P>
<P>New in this version is an algorithm for learning ranking functions [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
2002c</A>]. The goal is to learn a function from preference examples, so that it
orders a new set of objects as accurately as possible. Such ranking problems
naturally occur in applications like search engines and recommender systems.</P>
<P>Futhermore, this version includes an algorithm for training large-scale
transductive SVMs. The algorithm proceeds by solving a sequence of optimization
problems lower-bounding the solution using a form of local search. A detailed
description of the algorithm can be found in [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
1999c</A>]. A similar transductive learner, which can be thought of as a
transductive version of k-Nearest Neighbor is the <A
href="http://sgt.joachims.org/">Spectral Graph Transducer</A>. </P>
<P>SVM<I><SUP>light</I></SUP> can also train SVMs with cost models (see [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Morik et al.,
1999</A>]).</P>
<P>The code has been used on a large range of problems, including text
classification [<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
1999c</A>][<A
href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims,
1998a</A>], image recognition tasks, bioinformatics and medical applications.
Many tasks have the property of sparse instance vectors. This implementation
makes use of this property which leads to a very compact and efficient
representation.</P>
<H2>Source Code and Binaries</H2>
<P>The program is free for scientific use. Please contact me, if you are
planning to use the software for commercial purposes. The software must not be
further distributed without prior permission of the author. If you use
SVM<I><SUP>light</I></SUP> in your scientific work, please cite as </P>
<UL>
<LI>T. Joachims, Making large-Scale SVM Learning Practical. Advances in Kernel
Methods - Support Vector Learning, B. Sch鰈kopf and C. Burges and A. Smola
(ed.), MIT-Press, 1999. <BR><A
href="http://www.joachims.org/publications/joachims_99a.pdf"
target=_top>[PDF]</A><A
href="http://www.joachims.org/publications/joachims_99a.ps.gz"
target=_top>[Postscript (gz)]</A> </LI></UL>
<P>I would also appreciate, if you sent me (a link to) your papers so that I can
learn about your research. The implementation was developed on Solaris 2.5 with
gcc, but compiles also on SunOS 3.1.4, Solaris 2.7, Linux, IRIX, Windows NT, and
Powermac (after small modifications, see <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_light_faq.html">FAQ</A>).
The source code is available at the following location: </P>
<DIR>
<P><A href="http://download.joachims.org/svm_light/current/svm_light.tar.gz"
target=_top>http://download.joachims.org/svm_light/current/svm_light.tar.gz</A></P></DIR>
<P>If you just want the binaries, you can download them for the following
systems:</P>
<UL>
<LI>Solaris: <A
href="http://download.joachims.org/svm_light/current/svm_light_solaris.tar.gz"
target=_top>http://download.joachims.org/svm_light/current/svm_light_solaris.tar.gz</A>
<LI>Windows: <A
href="http://download.joachims.org/svm_light/current/svm_light_windows.zip"
target=_top>http://download.joachims.org/svm_light/current/svm_light_windows.zip</A>
<LI>Cygwin: <A
href="http://download.joachims.org/svm_light/current/svm_light_cygwin.tar.gz"
target=_top>http://download.joachims.org/svm_light/current/svm_light_cygwin.tar.gz</A>
<LI>Linux: <A
href="http://download.joachims.org/svm_light/current/svm_light_linux.tar.gz"
target=_top>http://download.joachims.org/svm_light/current/svm_light_linux.tar.gz</A>
</LI></UL>
<P><A href="mailto:thorsten@joachims.org">Please send me email</A> and let me
know that you got svm-light. I will put you on my mailing list to inform you
about new versions and bug-fixes. SVM<I><SUP>light</I></SUP> comes with a
quadratic programming tool for solving small intermediate quadratic programming
problems. It is based on the method of Hildreth and D'Espo and solves small
quadratic programs very efficiently. Nevertheless, if for some reason you want
to use another solver, the new version still comes with an interface to PR_LOQO.
The <A href="http://www.first.gmd.de/~smola/" target=_top>PR_LOQO optimizer</A>
was written by <A href="http://www.first.gmd.de/~smola/" target=_top>A.
Smola</A>. It can be requested from <A
href="http://www.kernel-machines.org/code/prloqo.tar.gz"
target=_top>http://www.kernel-machines.org/code/prloqo.tar.gz</A>. </P>
<H2>Installation</H2>
<P>To install SVM<I><SUP>light</I></SUP> you need to download
<TT>svm_light.tar.gz</TT>. Create a new directory:</P>
<DIR><TT>
<P>mkdir svm_light</P></TT></DIR>
<P>Move <TT>svm_light.tar.gz</TT> to this directory and unpack it with </P>
<DIR><TT>
<P>gunzip -c svm_light.tar.gz | tar xvf -</P></TT></DIR>
<P>Now execute </P>
<DIR><TT>
<P>make or make all</P></TT></DIR>
<P>which compiles the system and creates the two executables </P>
<DIR><TT>svm_learn (learning module)</TT><BR><TT>svm_classify (classification
module)</TT> </DIR>
<P>If you do not want to use the built-in optimizer but PR_LOQO instead, create
a subdirectory in the svm_light directory with </P>
<DIR><TT>
<P>mkdir pr_loqo</P></TT></DIR>
<P>and copy the files <TT>pr_loqo.c</TT> and <TT>pr_loqo.h</TT> in there. Now
execute </P>
<DIR><TT>
<P>make svm_learn_loqo</P></TT></DIR>
<P>If the system does not compile properly, check this <A
href="http://www.cs.cornell.edu/People/tj/svm_light/svm_light_faq.html">FAQ</A>.</P>
<H2>How to use</H2>
<P>This section explains how to use the SVM<I><SUP>light</I></SUP> software. A
good introduction to the theory of SVMs is Chris Burges' <A
href="http://www.kernel-machines.org/papers/Burges98.ps.gz"
target=_top>tutorial</A>. </P>
<P>SVM<I><SUP>light</I></SUP> consists of a learning module (<TT>svm_learn</TT>)
and a classification module (<TT>svm_classify</TT>). The classification module
can be used to apply the learned model to new examples. See also the examples
below for how to use <TT>svm_learn</TT> and <TT>svm_classify</TT>. </P><TT>
<P>svm_learn</TT> is called with the following parameters:</P>
<DIR><TT>
<P>svm_learn [options] example_file model_file</P></TT></DIR>
<P>Available options are: </P>
<DIR><PRE>General options:
-? - this help
-v [0..3] - verbosity level (default 1)
Learning options:
-z {c,r,p} - select between classification (c), regression (r), and
preference ranking (p) (see [<A href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims, 2002c</A>])
(default classification)
-c float - C: trade-off between training error
and margin (default [avg. x*x]^-1)
-w [0..] - epsilon width of tube for regression
(default 0.1)
-j float - Cost: cost-factor, by which training errors on
positive examples outweight errors on negative
examples (default 1) (see [<A href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Morik et al., 1999</A>])
-b [0,1] - use biased hyperplane (i.e. x*w+b0) instead
of unbiased hyperplane (i.e. x*w0) (default 1)
-i [0,1] - remove inconsistent training examples
and retrain (default 0)
Performance estimation options:
-x [0,1] - compute leave-one-out estimates (default 0)
(see [5])
-o ]0..2] - value of rho for XiAlpha-estimator and for pruning
leave-one-out computation (default 1.0)
(see [<A href="http://www.cs.cornell.edu/People/tj/svm_light/#References">Joachims, 2002a</A>])
-k [0..100] - search depth for extended XiAlpha-estimator
(default 0)
⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -