Covariance Functions

We implemented several different covariance functions (CFs) you can work with. To use GP-regression with these covariance functions it is highly recommended to model the noise of the data in one extra covariance function (pygp.covar.noise.NoiseISOCF) and add this noise CF to the CF you are calculating by putting them all together in one pygp.covar.combinators.SumCF.

For example to use the squared exponential CF with noise:

from pygp.covar import se, noise, combinators

#Feature dimension of the covariance: 
dimensions = 1

SECF = se.SEARDCF(dim)
noise = noise.NoiseISOCF()
covariance = combinators.SumCF((SECF,noise))

Covariance Function Combinators

Each combinator is a covariance function (CF) itself. It combines one or several covariance function(s) into another. For instance, pygp.covar.combinators.SumCF combines all given CFs into one sum; use this class to add noise.

class pygp.covar.combinators.SumCF(covars, *args, **kw_args)

Bases: pygp.covar.covar_base.CovarianceFunction

Sum Covariance function. This function adds up the given CFs and returns the resulting sum.

covars : [pygp.covar.CovarianceFunction]

Covariance functions to sum up.
K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters theta and inputs x1 and x2. The result will be the sum covariance of all covariance functions combined in this sum covariance.

Parameters: See pygp.covar.CovarianceFunction

Kgrad_theta(theta, x1, i)

The partial derivative of the covariance matrix with respect to i-th hyperparameter.

Parameters: See pygp.covar.CovarianceFunction

get_hyperparameter_names()

return the names of hyperparameters to make identification easier

class pygp.covar.combinators.ProductCF(covars, *args, **kw_args)

Bases: pygp.covar.covar_base.CovarianceFunction

Product Covariance function. This function multiplies the given CFs and returns the resulting product.

Parameters:

covars : [CFs of type pygp.covar.CovarianceFunction]

Covariance functions to be multiplied.
K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters theta and inputs x1 and x2. The result will be the product covariance of all covariance functions combined in this product covariance.

Parameters: See pygp.covar.CovarianceFunction

Kgrad_theta(theta, x, i)

The derivatives of the covariance matrix for the i-th hyperparameter.

Parameters: See pygp.covar.CovarianceFunction

get_hyperparameter_names()

return the names of hyperparameters to make identificatio neasier

class pygp.covar.combinators.ShiftCF(covar, replicate_indices, *args, **kw_args)

Bases: pygp.covar.covar_base.CovarianceFunction

Time Shift Covariance function. This covariance function depicts the time shifts induced by the data and covariance function given and passes the shifted inputs to the covariance function given. To calculate the shifts of the inputs make shure the covariance function passed implements the derivative after the input Kd_dx(theta, x).

covar : CF of type pygp.covar.CovarianceFunction

Covariance function to be used to depict the time shifts.

replicate_indices : [int]

The indices of the respective replicates, corresponding to the inputs. For instance: An input with three replicates:

/ rep1 rep2 rep3
input = [ -1,0,1,2, -1,0,1,2, -1,0,1,2]
replicate_indices = [ 0,0,0,0, 1,1,1,1, 2,2,2,2]

Thus, the replicate indices represent which inputs correspond to which replicate.

K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters theta and inputs x1 and x2. The result will be the covariance of the covariance function given, calculated on the shifted inputs x1,x2. The shift is determined by the last n_replicate parameters of theta, where n_replicate is the number of replicates this CF conducts.

Parameters:

theta : [double]
the hyperparameters of this CF. Its structure is as follows: [theta of covar, time-shift-parameters]

Others see pygp.covar.CovarianceFunction

Kgrad_theta(theta, x, i)

Get Covariance matrix K with given hyperparameters theta and inputs x1 and x2. The result will be the covariance of the covariance function given, calculated on the shifted inputs x1,x2. The shift is determined by the last n_replicate parameters of theta, where n_replicate is the number of replicates this CF conducts.

Parameters:

theta : [double]
the hyperparameters of this CF. Its structure is as follows:: [theta of covar, time-shift-parameters]
i : int
the partial derivative of the i-th hyperparameter shal be returned.
get_hyperparameter_names()

return the names of hyperparameters to make identificatio neasier

Squared Exponential Covariance functions

This class provides some ready-to-use implemented squared exponential covariance functions (SEs). These SEs do not model noise, so combine them by a pygp.covar.combinators.SumCF or pygp.covar.combinators.ProductCF with the pygp.covar.noise.NoiseISOCF, if you want noise to be modelled by this GP.

class pygp.covar.se.SqexpCFARD(*args, **kwargs)

Bases: pygp.covar.covar_base.CovarianceFunction

Standart Squared Exponential Covariance function.

Parameters:

  • dimension : int

    The dimension of this SE. For instance a 2D SE has hyperparameters like:

    covar_hyper = [Amplitude,1stD Length-Scale, 2ndD Length-Scale]
  • dimension_indices : [int]

    Optional: The indices of the n_dimensions in the input. For instance the n_dimensions of inputs are in 2nd and 4th dimension dimension_indices would have to be [1,3].

K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters and inputs X=x1 and X`*`=x2.

Parameters: See pygp.covar.CovarianceFunction

Kdiag(theta, x1)

Get diagonal of the (squared) covariance matrix.

Parameters: See pygp.covar.CovarianceFunction

Kgrad_theta(theta, x1, i)

The derivatives of the covariance matrix for each hyperparameter, respectively.

Parameters: See pygp.covar.CovarianceFunction

Kgrad_x(theta, x1, x2, d)

The partial derivative of the covariance matrix with respect to x, given hyperparameters theta.

Parameters: See pygp.covar.CovarianceFunction

get_hyperparameter_names()

return the names of hyperparameters to make identification easier

get_number_of_parameters()

Return the number of hyperparameters this CF holds.

Noise covariance function

NoiseCFISO NoiseCFReplicates

class pygp.covar.noise.NoiseCFISO(*args, **kw_args)

Bases: pygp.covar.covar_base.CovarianceFunction

Covariance function for Gaussian observation noise for all datapoints as a whole.

K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters theta and inputs args = X[, X’]. Note that this covariance function will only get noise as hyperparameter!

Parameters: See pygp.covar.CovarianceFunction

Kgrad_theta(theta, x1, i)

The derivative of the covariance matrix with respect to i-th hyperparameter.

Parameters: See pygp.covar.CovarianceFunction

Classes for linear covariance function

Linear covariance functions

LinearCFISO LinearCFARD

fixed covariance functions Classes for fixed covarinace functions ====================================== Linear covariance functions

FixedCF

class pygp.covar.CovarianceFunction(n_dimensions=1, dimension_indices=None)

Bases: object

Abstract super class for all implementations of covariance functions:

Important: All Covariance Functions have to inherit from this class in order to work properly with this GP framework.

Parameters:

n_dimensions : int

standard: n_dimension = 1. The number of dimensions (i.e. features) this CF holds.

dimension_indices : [int]

The indices of dimensions (features) this CF takes into account.
K(theta, x1, x2=None)

Get Covariance matrix K with given hyperparameters theta and inputs x1 and optional x2. If only x1 is given the covariance matrix is computed with x1 against x1.

Parameters:

theta : [double]

The hyperparameters for which the covariance matrix shall be computed. theta are the hyperparameters for the respective covariance function. For instance pygp.covar.se.SEARDCF holds hyperparameters as follows:

`[Amplitude, Length-Scale(s)]`.

x1 : [double]

The training input X, for which the pointwise covariance shall be calculated.

x2 : [double]

The interpolation input X`*`, for which the pointwise covariance shall be calculated.
Kdiag(theta, x1)

Get diagonal of the (squared) covariance matrix.

Default: Return the diagonal of the fully calculated Covariance Matrix. This may be overwritten more efficiently.

Kgrad_theta(theta, x1, i)

Get partial derivative of covariance matrix K with respect to the i-th given hyperparameter theta[i].

Parameters:

theta : [double]

The hyperparameters for covariance.

x1 : [double]

The training input X.

i : int

The index of the hyperparameter, which’s partial derivative shall be returned.
Kgrad_x(theta, x1, x2, d)

Partial derivatives of K[X1,X2] with respect to x1(:)^d RV: matrix of size [x1,x2] containin all values of d/dx1^{i,d} K(X1,X2)

Kgrad_xdiag(theta, x1, d)

Diagonal of partial derivatives of K[X1,X1] w.r.t. x1(:)^d RV: vector of size [x1] cotaining all partial derivatives d/dx1^{i,d} diag(K(X1,X2))

get_default_hyperparameters(x=None, y=None)

Return default hyperpameters.

Default:: No hyperparameters; Returns an empty array.

get_hyperparameter_names()

Return names of hyperparameters to make identification easier

get_n_dimensions()

Returns the number of dimensions, specified by user.

get_number_of_parameters()

Return number of hyperparameters, specified by user.

set_dimension_indices(active_dimension_indices=None)

Get the active_dimensions for this covariance function, i.e. the indices of the feature dimensions of the training inputs, which shall be used for the covariance.