readme

关于支持向量机的,有专门的工具箱,很好用,有什么问题请指教

Liblinear is a simple package for solving large-scale regularized
linear classification. It currently supports L2-regularized logistic
regression, L2-loss support vector machines, and L1-loss support
vector machines. This document explains the usage of liblinear.

Table of Contents
=================

- When to use LIBLINEAR but not LIBSVM
- Quick Start
- Installation
- `train' Usage
- `predict' Usage
- Examples
- Library Usage
- Building Windows Binaries
- Additional Information
- MATLAB interface

When to use LIBLINEAR but not LIBSVM
====================================

There are some large data for which with/without nonlinear mappings
gives similar performances.  Without using kernels, one can train a
much larger set via a linear classifier.  These data usually have a
large number of features. Document classification is an example.

For the software LIBSVM, please check
http://www.csie.ntu.edu.tw/~cjlin/libsvm


Quick Start
===========

See the section ``Installation'' for installing liblinear.

After installation, there are programs `train' and `predict' for
training and testing, respectively.

About the data format, please check the README file of libsvm.

A sample classification data included in this package is `heart_scale'.

Type `train heart_scale', and the program will read the training
data and output the model file `heart_scale.model'. If you have a test
set called heart_scale.t, then type `predict heart_scale.t
heart_scale.model output' to see the prediction accuracy. The `output'
file contains the predicted class labels.

For more information about `train' and `predict', see the sections
`train' Usage and `predict' Usage.

To obtain good performances, sometimes one needs to scale the
data. Please check the program `svm-scale' of LIBSVM. For large and
sparse data, use `-l 0' to keep the sparsity.

Installation
============

On Unix systems, type `make' to build the `train' and `predict'
programs. Run them without arguments to show the usages.

On other systems, consult `Makefile' to build them (e.g., see
'Building Windows binaries' in this file) or use the pre-built
binaries (Windows binaries are in the directory `windows').

This software uses some level-1 BLAS subroutines. The needed functions are 
included in this package.  If a BLAS library is available on your
machine, you may use it by modifying the Makefile: Unmark the following line

	#LIBS ?= -lblas

and mark 

	LIBS ?= blas/blas.a

`train' Usage
=============

Usage: train [options] training_set_file [model_file]
options:
-s type : set type of solver (default 1)
	0 -- L2-regulraized logistic regression
	1 -- L2-loss support vector machines (dual)	
	2 -- L2-loss support vector machines (primal)
	3 -- L1-loss support vector machines (dual)
-c cost : set the parameter C (default 1)
-e epsilon : set tolerance of termination criterion
	-s 0 and 2 
		|f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2, 
		where f is the primal function, (default 0.01)
	-s 1 and 3
		|min(max(alpha_i - G_i,0),C)-alpha_i|<= eps, 
		where G is the gradient of the dual, (default 0.1)
-B bias : if bias >= 0, instance x becomes [x; bias]; if < 0, no bias term added (default 1)
-wi weight: weights adjust the parameter C of different classes (see README for details)
-v n: n-fold cross validation mode

Formulations:

For L2-regularized logistic regression, we solve

min_w w^Tw/2 + C \sum log(1 + exp(-y_i w^Tx_i))

For L2-loss SVM, we solve

min_w w^Tw/2 + C \sum max(0, 1- y_i w^Tx_i)^2

For L2-loss SVM dual, we solve

min_alpha  0.5(alpha^T (Q + I/2/C) alpha) - e^T alpha 
    s.t.   0 <= alpha_i,

For L1-loss SVM dual, we solve

min_alpha  0.5(alpha^T Q alpha) - e^T alpha 
    s.t.   0 <= alpha_i <= C,

where

Q is a matrix with Q_ij = y_i y_j x_i^T x_j.

If bias >= 0, w becomes [w; w_{n+1}] and x becomes [x; bias].

We implement 1-vs-the rest multi-class strategy. In training i
vs. non_i, their C parameters are (weight from -wi)*C and C,
respectively. If there are only two classes, we train only one
model. Thus weight1*C vs. weight2*C is used. See examples below.

`predict' Usage
===============

Usage: predict [options] test_file model_file output_file
options:
-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0)

Examples
========

> train data_file

Train linear SVM with L2-loss function.

> train -s 0 data_file

Train a logistic regression model.

> train -v 5 -e 0.001 data_file

Do five-fold cross-validation using L2-loss svm.
Use a smaller stopping tolerance 0.001 than the default
0.1 if you want more accurate solutions.

> train -c 10 -w1 2 -w2 5 -w3 2 four_class_data_file

Train four classifiers:
positive	negative	Cp	Cn
class 1		class 2,3,4.	20	10
class 2		class 1,3,4.	50	10
class 3		class 1,2,4.	20	10
class 4		class 1,2,3.	10	10

> train -c 10 -w3 1 -w2 5 two_class_data_file

If there are only two classes, we train ONE model. 
The C values for the two classes are 10 and 50. 

> predict -b 1 test_file data_file.model output_file

Output probability estimates (for logistic regression only).

Library Usage
=============

- Function: model* train(const struct problem *prob,
                const struct parameter *param);

    This function constructs and returns a linear classification model
    according to the given training data and parameters.

    struct problem describes the problem:

        struct problem
        {
            int l, n;
            int *y;
            struct feature_node **x;
            double bias;
        };

    where `l' is the number of training data. If bias >= 0, we assume
    that one additional feature is added to the end of each data
    instance. `n' is the number of feature (including the bias feature
    if bias >= 0). `y' is an array containing the target values. And
    `x' is an array of pointers,
    each of which points to a sparse representation (array of feature_node) of one
    training vector.

    For example, if we have the following training data:

    LABEL       ATTR1   ATTR2   ATTR3   ATTR4   ATTR5
    -----       -----   -----   -----   -----   -----
    1           0       0.1     0.2     0       0
    2           0       0.1     0.3    -1.2     0
    1           0.4     0       0       0       0
    2           0       0.1     0       1.4     0.5
    3          -0.1    -0.2     0.1     1.1     0.1

    and bias = 1, then the components of problem are:

    l = 5
    n = 6

    y -> 1 2 1 2 3

    x -> [ ] -> (2,0.1) (3,0.2) (6,1) (-1,?)
         [ ] -> (2,0.1) (3,0.3) (4,-1.2) (6,1) (-1,?)
         [ ] -> (1,0.4) (6,1) (-1,?)
         [ ] -> (2,0.1) (4,1.4) (5,0.5) (6,1) (-1,?)
         [ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (6,1) (-1,?)

    struct parameter describes the parameters of a linear classification model:

	struct parameter
	{
		int solver_type;

		/* these are for training only */
		double eps;	        /* stopping criteria */
		double C;
		int nr_weight;
		int *weight_label;
		double* weight;
	};

    solver_type can be one of L2_LR, L2LOSS_SVM_DUAL, L2LOSS_SVM, L1LOSS_SVM_DUAL.

    L2_LR		L2-regularized logistic regression
    L2LOSS_SVM_DUAL	L2-loss support vector machines (dual) 
    L2LOSS_SVM		L2-loss support vector machines (primal)
    L1LOSS_SVM_DUAL	L1-loss support vector machines (dual) 	

    C is the cost of constraints violation. (we usually use 1 to 1000)
    eps is the stopping criterion. (we usually use 0.01).

    nr_weight, weight_label, and weight are used to change the penalty
    for some classes (If the weight for a class is not changed, it is
    set to 1). This is useful for training classifier using unbalanced
    input data or with asymmetric misclassification cost.

    nr_weight is the number of elements in the array weight_label and
    weight. Each weight[i] corresponds to weight_label[i], meaning that
    the penalty of class weight_label[i] is scaled by a factor of weight[i].

    If you do not want to change penalty for any of the classes,
    just set nr_weight to 0.

    *NOTE* To avoid wrong parameters, check_parameter() should be
    called before train().

- Function: void cross_validation(const problem *prob, const parameter *param, int nr_fold, double *target);

    This function conducts cross validation. Data are separated to
    nr_fold folds. Under given parameters, sequentially each fold is
    validated using the model from training the remaining. Predicted
    labels in the validation process are stored in the array called
    target.

    The format of prob is same as that for train().

- Function: int predict(const model *model_, const feature_node *x);

    This functions classifies a test vector using the given
    model. The predicted label is returned.

- Function: int predict_values(const struct model *model_,
            const struct feature_node *x, double* dec_values);

    This function gives nr_classifier decision values in the array
    dec_values. nr_classifier is 1 if there are two classes, and is
    the number of classes otherwise. 

    We use one-vs-the rest multi-class strategy. The class with the
    highest decision values is returned.

- Function: int predict_probability(const struct model *model_,
            const struct feature_node *x, double* prob_estimates);

    This function gives nr_class probability estimates in the array
    prob_estimates. nr_class can be obtained from the function
    get_nr_class. The class with the highest probability is
    returned. Currently, we support only the probability outputs of
    logistic regression.

- Function: int get_nr_feature(const model *model_);

    The function gives the number of attributes of the model.

- Function: int get_nr_class(const model *model_);

    The function gives the number of classes of the model.

- Function: void get_labels(const model *model_, int* label);

    This function outputs the name of labels into an array called label.

- Function: const char *check_parameter(const struct problem *prob,
            const struct parameter *param);

    This function checks whether the parameters are within the feasible
    range of the problem. This function should be called before calling
    train() and cross_validation(). It returns NULL if the
    parameters are feasible, otherwise an error message is returned.
	
- Function: int save_model(const char *model_file_name,
            const struct model *model_);

    This function saves a model to a file; returns 0 on success, or -1
    if an error occurs.

- Function: struct model *load_model(const char *model_file_name);

    This function returns a pointer to the model read from the file,
    or a null pointer if the model could not be loaded.

- Function: void destroy_model(struct model *model_);

    This function frees the memory used by a model.

- Function: void destroy_param(struct parameter *param);

    This function frees the memory used by a parameter set.

Building Windows Binaries
=========================

Windows binaries are in the directory `windows'. To build them via
Visual C++, use the following steps:

1. Open a dos command box and change to liblinear directory. If
environment variables of VC++ have not been set, type

"C:\Program Files\Microsoft Visual Studio 8\VC\bin\vcvars32.bat"

You may have to modify the above according which version of VC++ or
where it is installed.

2. Type

nmake -f Makefile.win clean all


MATLAB Interface
================

Please check the file README in the directory `matlab'.

Additional Information
======================

If you find LIBLINEAR helpful, please cite it as

C.-J. Lin, R. C. Weng, and S. S. Keerthi.
Trust region Newton method for large-scale logistic
regression. Technical report, 2007. A short version appears
in ICML 2007. Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear

For any questions and comments, please send your email to
[e-mail]