readme

一个计算线性支持向量机的matlab源代码

Two things are provided in this directory: an automatic model selection
tool and a python binding for libsvm.

		Part I: Model Selection Tools

Introduction
===============

grid.py is a model selection tool for C-SVM classification using RBF
(radial basis function) kernel. It uses cross validation (CV) technique
to estimate the accuracy of each parameter combination in the specified 
range and helps you to decide the best parameters for your problem.

grid.py directly executes libsvm binaries (so no python binding is needed)
for cross validation and then draw contour of CV accuracy using gnuplot.
You must have libsvm and gnuplot installed before using it. The package 
gnuplot is available at http://www.gnuplot.info/

Usage: grid.py [-log2c begin,end,step] [-log2g begin,end,step] [-v fold] 
       [-svmtrain pathname] [-gnuplot pathname] [-out pathname] [-png pathname]  
       [additional parameters for svm-train] dataset

The program conducts v-fold cross validation using parameter C (and gamma) 
= 2^begin, 2^(begin+step), ..., 2^end. 

You can specify where the libsvm executable and gnuplot are using the
-svmtrain and -gnuplot parameters.

For windows users, please use pgnuplot.exe. If you are using gnuplot
3.7.1, please upgrade to version 3.7.3. The version 3.7.1 has a bug.

Example
=======

> python grid.py -log2c -5,5,1 -log2g -4,0,1 -v 5 -m 300 heart_scale

Users (in particular MS Windows users) may need to specify the path of
executable files. You can either change paths in the beginning of
grid.py or specify them in the command line. For example,

> grid.py -log2c -5,5,1 -svmtrain c:\libsvm-2.7\windows\svmtrain.exe -gnuplot c:\tmp\gnuplot\bin\pgnuplot.exe -v 10 heart_scale

Output: two files
dataset.png: the contour plot of the CV accuracy (generated by gnuplot)
dataset.out: the log of accuracy at each (C,gamma)

Parallel grid search (experimental)
===================================

You can conduct a parallel grid search by dispatching jobs to a cluster of 
computers which share the same file system. First, you add machine names 
in grid.py:

telnet_workers = ["linux1", "linux5", "linux5"]

The same machine (e.g., linux5 here) can be listed more than once if
it has multiple CPUs or has more RAM. If the local machine is the
best, you can also enlarge the nr_local_worker. For example:

nr_local_worker = 2

Example:

> python grid.py heart_scale
Password: ********

login ok linux1
login ok linux5
login ok linux5
.
.
.

The password is the one used for entering your system. If -log2c, -log2g, or
-v is not specified, default values are used. 

If your system uses ssh instead of telnet, you should setup ssh first so that
the authentication works without asking a password, and list the computer names
in ssh_workers.

		Part II: Python-to-libsvm interface

Introduction
============

Python (http://www.python.org/) is a programming language suitable for
rapid development. This python-to-libsvm interface is developed so 
users can easily experiment with libsvm using python. The interface is 
developed with SWIG, 

The original idea and the SWIG interface file was provided by Carl Staelin
([e-mail]) from HP Labs. The interface was integrated into the
libsvm package by Li-lun Wang ([e-mail]) from National Taiwan
University. Chih-Chung Chang ([e-mail]) from National
Taiwan University also contributed a lot of useful suggestions and help.

Installation
============
The build process for the various Unix systems is as follows:

Before you build the module, you need to find out the python include
directory, which is typically located at /usr/local/include/python2.3 or
/usr/include/python. You can set the variable PYTHON_INCLUDEDIR in
Makefile manually or use something like the following:

	make PYTHON_INCLUDEDIR=/usr/include/python all

Although the interface is generated by SWIG, it is not necessary to
have SWIG installed because the generated svmc_wrap.c is included in
this package (It was generated using SWIG 1.3.21). If you prefer
generating the interface with SWIG on your own, you can simply remove
the generated files with

	make moreclean

before building the module.

When the build process completes, a shared object called svmc.so will be
created.

For win32 systems, the shared library svmc.dll is ready in the
directory windows/python. You need to copy it to this directory.  The
dll file depends on different versions of python, so you may have to
re-make it by following the instruction of building windows binaries
in libsvm README.

Usage
=====
To use the module, the files svm.py and the shared library (namely svmc.so
or svmc.dll) must be placed in the current directory, the python library
directory, or the directory where the environment variable PYTHONPATH
points to. The user then imports everything in svm.py to use libsvm in
python:

	from svm import *

There are three classes in svm.py, namely svm_parameter, svm_problem, and
svm_model.

svm_parameter is used to set the parameters of the training
process. The attributes in svm_parameter include svm_type,
kernel_type, degree, gamma, coef0, nu, cache_size, C, eps, p,
shrinking, nr_weight, weight_label, and weight. Available svm types
include C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, and NU_SVR. Available
kernel types include LINEAR, POLY, RBF, and SIGMOID. The user can
setup the parameters with the constructor and keyword arguments:

	param = svm_parameter(kernel_type = LINEAR, C = 10)

The user can also modify the parameters later:

	param.kernel_type = RBF

svm_problem is used to hold the training data for the problem. The
constructor takes two arguments; the first of them is the list of labels,
and the other is the list of samples. For example

	prob = svm_problem([1,-1],[[1,0,1],[-1,0,-1]])

or equivalently

	prob = svm_problem([1,-1],[{1:1,3:1},{1:-1,3:-1}])

Once the parameter and problem are ready, we can construct the model:

	m = svm_model(prob, param)

To conduct n-fold cross validation; predicted labels in the validation
process are returned.

	target = cross_validation(prob, param, n)

To predict a new sample with the model:

	r = m.predict([1, 1, 1])

To obtain decision values of predicting a sample:

	d = m.predict_values([1, 1, 1])	

To predict a new sample and obtain probability estimates;
return value is a dict that maps labels to probabilities.

	prd, prb = m.predict_probability([1, 1, 1])
	sample of prd : 1.0
	sample of prb : {1:0.6, -1:0.4}

To obtain sigma of the probability density function for regression;
see ../README for the definition of the function.

	sigma = m.get_svr_probability()

To obtain the probability density function for regression; see
../README for the definition of the function.

	pdf = m.get_svr_pdf()
	probability = pdf(z)

To save the model to a file:

	m.save('test.model')

and to load the model from a file:

	m = svm_model('test.model')

Examples
========

There are two examples in this package. The one is svm_test.py, and the
other is test_cross_validation.py.

svm_test.py tests various kernels on a three-class problem with
C-SVM. It also demonstrates how to obtain decision values and
probability estimates.

test_cross_validation.py demonstrates loading data from a file and
does a ten-fold cross validation on the heart_scale dataset. It makes
use of cross_validation.py which calls the C++ cross validation
subroutine.