📄 regression.html
字号:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html> <head> <title>Gaussian Process Regression</title> <link type="text/css" rel="stylesheet" href="style.css"> </head> <body><h2>Gaussian Process Regression</h2>This page contains 4 sections<ol><li> <a href="#desc">Description</a> of the GP regression function<tt>gpr.m</tt></li> <li> <a href="#1-d">1-d demo</a> using gpr.m</li> <li> <ahref="#ard">Multi-dimensional demo</a> using gpr.m with Automatic RelevanceDetermination (ARD)</li> <li> <a href="#ref">References</a></li></ol><h3 id="desc">1. Description of the GP regression function <tt>gpr.m</tt></h3>The basic computations needed for standard Gaussian process regression (GPR)are straight forward to implement in matlab. Several implementations arepossible, here we present an implementation closely resembling Algorithm 2.1(p. 19) with three exceptions: Firstly, the predicitive variance returned isthe variance for noisy test-cases, whereas Algorithm 2.1 gives the variance forthe noise-free latent function; conversion between the two variances is done bysimply adding (or subtracting) the noise variance. Secondly, the<em>negative</em> log marginal likelihood is returned, and thirdly the partialderivatives of the negative log marginal likelihood, equation (5.9), section5.4.1, page 114 are also computed.</p>Algorithm 2.1, section 2.2, page 19:</p><center><img src="alg21.gif"></center><br>A simple implementation of a Gaussian process for regression is provided by the<a href="../gpml/gpr.m">gpr.m</a> program (which can conveniently be usedtogether with <a href="../gpml/minimize.m">minimize.m</a> for optimization ofthe hyperparameters). The program can do one of two things:<ol><li> compute the negative log marginal likelihood and its partial derivativeswrt. the hyperparameters, usage</p><pre>[nlml dnlml] = gpr(logtheta, covfunc, x, y)</pre>which is used when "training" the hyperparameters, or</li><br><li> compute the (marginal) predictive distribution of test inputs, usage</p><pre>[mu S2] = gpr(logtheta, covfunc, x, y, xstar)</pre></li></ol>Selection between the two modes is indicated by the presence (or absence) oftest cases, <tt>xstar</tt>. The arguments to the <ahref="../gpml/gpr.m">gpr.m</a> function are:<br><table border=0 cols=2 width="100%"><tr><td><b>inputs</b></td><td></td></tr><tr><td width="10%"><tt>logtheta</tt></td><td>a (column) vector containing thelogarithm of the hyperparameters</td></tr><tr><td><tt>covfunc</tt></td><td>the covariance function</td></tr><tr><td><tt>x</tt></td><td>a <tt>n</tt> by <tt>D</tt> matrix of traininginputs</td></tr><tr><td><tt>y</tt></td><td>a (column) vector if training settargets (of length <tt>n</tt>)</td></tr><tr><td><tt>xstar</tt></td><td>a <tt>nn</tt> by <tt>D</tt> matrix of testinputs</td></tr><tr><td><b>outputs</b></td><td></td></tr><tr><td><tt>nlml</tt></td><td>the negative log marginal likelihood</td></tr><tr><td><tt>dnlml</tt></td><td>(column) vector with the partial derivatives of the negative log marginal likelihood wrt. thelogarithm of the hyperparameters.</td></tr><tr><td><tt>mu</tt></td><td>(column) vector (of length <tt>nn</tt>) ofpredictive means</td></tr><tr><td><tt>S2</tt></td><td>(column) vector (of length <tt>nn</tt>) ofpredictive variances</td></tr></table><br>The <tt>covfunc</tt> argument specifies the function to be called to evaluatecovariances. The <tt>covfunc</tt> can be specified either as a string namingthe function to be used or as a cell array. A number of covariance functionsare provided, see <a href="../gpml/covFunctions.m">covFunctions.m</a> for amore complete description. A commonly used covariance function is the sum ofa sqaured exponential (SE) contribution and independent noise. This can bespecified as:<pre> covfunc = {'covSum', {'covSEiso','covNoise'}};</pre>where <tt>covSum</tt> merely does some bookkeeping and calls the squaredexponential (SE) covariance function <ahref="../gpml/covSEiso.m">covSEiso.m</a> and the independent noise covariance<a href="../gpml/covNoise.m">covNoise.m</a>. The squared exponential (SE) covariance function (also called the radial basisfunction (RBF) or Gaussian covariance function) is given by in equation (2.31)in section 2.3 page 19 for the scalar input case, and equation (5.1) section5.1 page 106 for multivariate inputs.<h3 id="1-d">2. 1-d demo using gpr.m</h3>You can either follow the example below or run a short script <ahref="../gpml-demo/demo_gpr.m">demo_gpr.m</a>.</p>First, add the directory containing the <tt>gpml</tt> code to your path(assuming that your current directory is <tt>gpml-demo</tt>:<pre> addpath('../gpml')</pre>Generate some data from a noisy Gaussian process (it is not essential tounderstand the details of this):<pre> n = 20; rand('state',18); randn('state',20); covfunc = {'covSum', {'covSEiso','covNoise'}}; loghyper = [log(1.0); log(1.0); log(0.1)]; x = 15*(rand(n,1)-0.5); y = chol(feval(covfunc{:}, loghyper, x))'*randn(n,1); % Cholesky decomp.</pre>Take a look at the data:<pre> plot(x, y, 'k+', 'MarkerSize', 17)</pre><center><img src="figl.gif"></center><br>Now, we compute the predictions for this data using a Gaussian process witha covariance function being the sum of a squared exponential (SE) contributionand an independent noise contribution:<pre> covfunc = {'covSum', {'covSEiso','covNoise'}};</pre>To verify how many hyperparameters this covariance function expects:<pre> feval(covfunc{:})</pre>which reurns <tt>'2+1'</tt>, ie two hyperparameters for the SE part (acharacteristic length-scale parameter and a signal magnitude parameter) and asingle parameter for the noise (noise magnitude). We specify thehyperparameters <tt>ell=1.0</tt> (characteristic length-scale),<tt>sigma2f=1.0</tt> (signal variance) and <tt>sigma2n=0.01</tt> (noisevariance)<pre> loghyper = [log(1.0); log(sqrt(1.0)); log(sqrt(0.01))]; % this is for figure 2.5(a)</pre>We place <tt>201</tt> test points evenly distributed in the interval <tt>[-7.5,7.5]</tt> and make predictions<pre> xstar = linspace(-7.5,7.5,201)'; [mu S2] = gpr(loghyper, covfunc, x, y, xstar);</pre>The predictive variance is computed for the distribution of test targets, whichare assumed to be as noisy as the training targets. If we are interestedinstead in the distribution of the noise-free function, we subtract the noisevariance from <tt>S2</tt><pre> S2 = S2 - exp(2*loghyper(3));</pre>To plot the mean and two standard error (95% confidence), noise-free, pointwiseerrorbars:<pre> hold on errorbar(xstar, mu, 2*sqrt(S2), 'g');</pre>Alternatively, you can also display the error range in gray-scale:<pre> clf f = [mu+2*sqrt(S2);flipdim(mu-2*sqrt(S2),1)]; fill([xstar; flipdim(xstar,1)], f, [7 7 7]/8, 'EdgeColor', [7 7 7]/8); hold on plot(xstar,mu,'k-','LineWidth',2); plot(x, y, 'k+', 'MarkerSize', 17);</pre>which procudes<br><center><img src="figl1.gif"></center><br>reproducing Figure 2.5(a), section 2.3, page 20. Note that the two standarddeviation error bars are for the noise-free function. If you rather want the95% confidence region for the noisy observations you should not subtract thenoise variance <tt>sigma2n</tt> from the predictive variances <tt>S2</tt>. Toreproduce Figs 2.5 (b) and (c) change the log hyperparameters when calling theprediction program to:<pre> loghyper = [log(0.3); log(1.08); log(5e-5)]; % this is for figure 2.5(b) loghyper = [log(3.0); log(1.16); log(0.89)]; % this is for figure 2.5(c)</pre>respectively. To learn, or optimize the hyperparameters by maximizing themarginal likelihood using the derivatives from equation (5.9) section 5.4.1page 114 using <a href="../gpml/minimize.m">minimize.m</a> do:<pre> loghyper = minimize([-1; -1; -1], 'gpr', -100, covfunc, x, y); exp(loghyper)</pre>Here, we initialize the log of the hyperparameters to be all at minus one. Weminimize the negative log marginal likelihood wrt. the log of thehyperparameters (allowing <tt>100</tt> evaluations of the function (this ismore than adequate)), and report the hyperparameters:<pre> 1.3659⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -