is there any simple code for beginners where i can experiment diff kernels used in gaussian process example in scikit learn to know their functions?

问题

actually i want to understand the kernels used in scikit learn gaussian example but i have zero knowledge about how those kernel behaves and when to use which and i also not getting any sample basic template code where i can use those kernel one by one and understand.The partial code is given below:

X, y = load_mauna_loa_atmospheric_co2()

Kernel with parameters given in GPML book

k1 = 66.0**2 * RBF(length_scale=67.0)  # long term smooth rising trend
k2 = 2.4**2 * RBF(length_scale=90.0) \
    * ExpSineSquared(length_scale=1.3, periodicity=1.0)  # seasonal component
# medium term irregularity
k3 = 0.66**2 \
    * RationalQuadratic(length_scale=1.2, alpha=0.78)
k4 = 0.18**2 * RBF(length_scale=0.134) \
    + WhiteKernel(noise_level=0.19**2)  # noise terms
kernel_gpml = k1 + k2 + k3 + k4

gp = GaussianProcessRegressor(kernel=kernel_gpml, alpha=0,
                              optimizer=None, normalize_y=True)
gp.fit(X, y)

print("GPML kernel: %s" % gp.kernel_)
print("Log-marginal-likelihood: %.3f"
      % gp.log_marginal_likelihood(gp.kernel_.theta))

# Kernel with optimized parameters
k1 = 50.0**2 * RBF(length_scale=50.0)  # long term smooth rising trend
k2 = 2.0**2 * RBF(length_scale=100.0) \
    * ExpSineSquared(length_scale=1.0, periodicity=1.0,
                     periodicity_bounds="fixed")  # seasonal component
# medium term irregularities
k3 = 0.5**2 * RationalQuadratic(length_scale=1.0, alpha=1.0)
k4 = 0.1**2 * RBF(length_scale=0.1) \
    + WhiteKernel(noise_level=0.1**2,
                  noise_level_bounds=(1e-3, np.inf))  # noise terms
kernel = k1 + k2 + k3 + k4

gp = GaussianProcessRegressor(kernel=kernel, alpha=0,
                              normalize_y=True)
gp.fit(X, y)

print("\nLearned kernel: %s" % gp.kernel_)
print("Log-marginal-likelihood: %.3f"
      % gp.log_marginal_likelihood(gp.kernel_.theta))

X_ = np.linspace(X.min(), X.max() + 30, 1000)[:, np.newaxis]
y_pred, y_std = gp.predict(X_, return_std=True)

# Illustration
plt.scatter(X, y, c='k')
plt.plot(X_, y_pred)
plt.fill_between(X_[:, 0], y_pred - y_std, y_pred + y_std,
                 alpha=0.5, color='k')
plt.xlim(X_.min(), X_.max())
plt.xlabel("Year")
plt.ylabel(r"CO$_2$ in ppm")
plt.title(r"Atmospheric CO$_2$ concentration at Mauna Loa")
plt.tight_layout()
plt.show()

回答1:

All the details are in the book by Rasmussen and Williams. The example you show is in Chapter 5 along with a detailed explanation of all the kernels used. They also show many examples of covariance functions and corresponding random functions.

I am not aware of a code to simply visualise various kernels but one can visualise the popular squared exponential function, which appears multiple times in the Mauna Loa example with different length scales, as follows:

import numpy as np
import matplotlib.pyplot as plt

def k_se(r,l):
    return np.exp(-r*r/(2*l*l)) 

r = np.arange(0.1,4,0.01)
plt.figure()
for ll in l:
    plt.plot(r,k_se(r,ll),label='length='+str(np.round(ll,1)))
    plt.xlabel('r')
    plt.ylabel('Covariance k(r)')
plt.legend(frameon=False)

Different kernels for different length scales look like:

However what's more interesting is drawing random functions from a gaussian process that has given covariance function. The following code is not meant for efficiency or speed but to make it easy to visualise these random functions.

def k_se_p(x1, x2, l):
    return np.exp(-((x1-x2)*(x1-x2))/(2*l*l))

def gm(x,l):
    return [[k_se_p(i,j,l) for j in x] for i in x]

x = np.arange(0.1,8,0.01)

It is instructive to first draw functions from the same length scale:

plt.figure() 
for i in range(5):
    ys = np.random.multivariate_normal(np.zeros(len(x)), gm(x,l[0]))
    if i==0:
        plt.plot(x,ys,color='blue',label='length='+str(np.round(l[0],1)))
    else:
        plt.plot(x,ys,color='blue')
    plt.xlabel('x')
    plt.ylabel('f(x)') 
plt.legend(frameon=False)

Which gives a not very smooth functions:

A larger length scale gives smoother functions:

plt.figure() 
for i in range(5):
    ys = np.random.multivariate_normal(np.zeros(len(x)), gm(x,l[-1]))
    if i==0:
        plt.plot(x,ys,color='magenta',label='length='+str(np.round(l[-1],1)))
    else:
        plt.plot(x,ys,color='magenta')
    plt.xlabel('x')
    plt.ylabel('f(x)')
plt.legend(frameon=False)

Finally we can draw one function from each length scale and plot them together:

plt.figure() 
for ll in l:
    ys = np.random.multivariate_normal(np.zeros(len(x)), gm(x,ll))
    plt.plot(x,ys,label='length='+str(np.round(ll,1)))
    plt.xlabel('x')
    plt.ylabel('f(x)') 
plt.legend(frameon=False)

来源：https://stackoverflow.com/questions/60772051/is-there-any-simple-code-for-beginners-where-i-can-experiment-diff-kernels-used

标签

scikit-learn

kernel

gaussian