readme

it is regression Algorithm in C/C++.

mpr - multivariate polynomial regression

This file provides some explanations on how to use the programs mpr
and mpx to compute and execute a regression polynomial. However, it
does not explain all options of the program. For a list of options,
call mpr and mpx without any arguments.

Enjoy,
Christian Borgelt

e-mail: [e-mail]
WWW:    http://www.borgelt.net/

------------------------------------------------------------------------

In this directory (regress/ex) you can find three very small datasets,
test1.tab to test3.tab, which can be used to test the programs mpr and
mpx. The first two are suited to determine a simple regression line,
the first also to determine a regression parabola. The third is meant
to demonstrate logistic regression.

To process the first data file, type
  mpr test.dom test1.tab test1.reg
This determines a regression line. The exact result is
  y = 43/35 x +68/35.
That is, the output of the program should be
  y = 1.22857 x +1.94286.
This function is written in the following form to the file test1.reg
(which also contains domain descriptions at the beginning):

polynomial(y) = {
  degree = 1;
  coeffs = { +1.22857, +1.94286 };
};

To keep track of the coefficients if more than one input and/or a
higher degree of the regression polynomial is used, the exponents of
the different input variables can be printed after each coeeficient.
This is achieved with the option -e, i.e., type
  mpr -e test.dom test1.tab test1.reg
Then the contents of the file test1.reg is:

polynomial(y) = {
  degree = 1;
  coeffs = {
    +1.22857 /* 1 */,
    +1.94286 /* 0 */ };
};

This means that +1.22857 is the coefficient of x^1 and +1.94286 is
the coefficient of x^0.

For the second data file, to be processed with
  mpr test.dom test2.tab test2.reg
the exact result is
  y = -27/26 x +30/13.
That is, the output of the program should be
  y = -1.03846 x +2.30769.
Or, in the output format of the program,

polynomial(y) = {
  degree = 1;
  coeffs = { -1.03846, +2.30769 };
};

To determine a regression parabola for the data vectors in the first
data file, type
  mpr -x2 test.dom test1.tab test1.qrg
The option -x2 specifies that a polynomial with degree 2 is desired.
The exact result is
  y = 15/44 x^2 +281/220 x +64/55.
That is, the output of the program should be
  y = +0.340909 x^2 +1.27727 x +1.16364.

polynomial(y) = {
  degree = 2;
  coeffs = { +0.340909, +1.27727, +1.16364 };
};

The third data file contains data vectors for logistic regression with
Y = 6 (limiting value for the logistic function to be found). It can
be processed with
  mpr -l6 test.dom test3.tab test3.reg
The output of the program (due to the natural logarithm in the
logit transformation it is not possible to give an exact result with
fractions) should be
  y = -1.37751 x +4.13253
That is, the logistic function is
  z = 6 /(1 +e^y) = 6 /(1 +e^{-1.37751 x +4.13253})
In the output format of the program, this looks like this:

polynomial(y) = {
  degree = 1;
  logit  = 6;
  coeffs = { -1.37751, +4.13253 };
};

Note the additional line starting with "logit", which states the
maximum for the logistic regression.

In addition to these simple tests, you can compile the program regdat.c
with "make regdat". This program generates 10000 vectors with three
regressor variables taking their values uniformly distributed in [-5,5]
and a response variable y that is computed as
  f(x,y,z) = 3 x^4 - 5 y^3 -2 z^3 +7 xy.
Invoking regdat and piping its output to mpr with
  regdat | mpr -x4 regdat.dom -
(the option -x4 specifies that a polynomial of degree 4 is desired)
should yield the function shown above, even though some additional
coefficients are printed. The values of these additional coefficients
should be very small, though (less than 1e-6), since they result from
roundoff errors.

With the program mpx an induced regression polynomial can be executed
on (new) data. Simply type
  mpx test1.reg test1.tab test1.out
to execute the first regression function determined above on the input
data file. This yields the output
  sse : 1.54286
  mse : 0.385714
  rmse: 0.621059
to the terminal, stating the sum of squared errors (sse), mean of the
squared errors (mse) and the root of the mean squared error (rmse).
In addition, the output file test1.out contains an additional column
with the values computed from the regression polynomial:
  x y reg
 -2 0 -0.51428
  0 1 1.94286
  1 3 3.17143
  2 5 4.4

------------------------------------------------------------------------

It should be noted that the sum of squared errors that is printed
in the log message of the regression computation is calculated in a
numerically unstable way and that therefore the value is unreliable.
Furthermore, for logistic regression this value is the sum of squared
errors w.r.t. the transformed coordinates, not the original ones.
If you desire to know the sum of squared errors, execute the program
mpx on the input data file, which also computes the mean squared error
as well as its root (see above).