机器学习-Random Sample Consensus Regression(RANSAC)回归

Section I: Brief Introduction on RANSAC

Linear regression models can be heavily impacted by the presence of outliers. In certain situations, a very small subset of our data can have a big effect on the estimated model coefficients. There are mant statistical tests that can be used to detect outliers. However, removing outliers always requiresour own judgement as data scientists as well as domain knowledge.

As an alternative to throwing out outliers, a robust method of regression using the random sample consensus (RANSAC) algorithm, which fits a regression model to a subset of the data, the so-called inliers. The iterative algorithm can be summarized as follows:

Step 1: Select a random number of samples to be inliers and fit the model
Step 2: Test all other data points against the fitted model and add those points that fall within a user-given tolerance to the inliers
Step 3: Refit the model using all inliers
Step 4: Estimate the error of the fitted model versus the inliers
Step 5: Terminate the algorithm if the performance meets a certain user-defined threshold or if a fixed number of iterations were reached; go back to Step 1 otherwise.

FROM
Sebastian Raschka, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.

代码

from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import mean_squared_error,r2_score
import matplotlib.pyplot as plt
import numpy as np
import warnings
warnings.filterwarnings("ignore")

plt.rcParams['figure.dpi']=200
plt.rcParams['savefig.dpi']=200
font = {'weight': 'light'}
plt.rc("font", **font)

#Section 1: Load data
price=datasets.load_boston()
X=price.data[:,5]
y=price.target

#Section 2: Wrap linear model in the RANSAC algorithm
from sklearn.linear_model import LinearRegression
from sklearn.linear_model import RANSACRegressor
from sklearn.metrics import mean_squared_error,r2_score

lr=LinearRegression()
lr.fit(X[:,np.newaxis],y)
y_pred=lr.predict(X.reshape(-1,1))
mse_score_lr=mean_squared_error(y,y_pred)
r2_score_lr=r2_score(y,y_pred)
print("MSE: %.3f, R2_Score: % .3f Via Linear Regression Model" % (mse_score_lr,r2_score_lr))

ransac=RANSACRegressor(LinearRegression(),
                       max_trials=100,
                       min_samples=50,
                       loss='absolute_loss',
                       residual_threshold=5.0,
                       random_state=0)

ransac.fit(X.reshape(-1,1),y)
y_pred=ransac.predict(X.reshape(-1,1))
mse_score_ransac=mean_squared_error(y,y_pred)
r2_score_ransac=r2_score(y,y_pred)
print("MSE: %.3f, R2_Score: % .3f Via RANSAC" % (mse_score_ransac,r2_score_ransac))

#Section 3: Visualize inliers and outliers from RANSAC-linear regression model
inlier_mask=ransac.inlier_mask_
outlier_mask=np.logical_not(inlier_mask)
line_X=np.arange(3,10,1)
line_y_ransac=ransac.predict(line_X[:,np.newaxis])

plt.scatter(X[inlier_mask],y[inlier_mask],c='steelblue',edgecolors='white',
            marker='o',label='Inliers')
plt.scatter(X[outlier_mask],y[outlier_mask],c='limegreen',edgecolors='white',
            marker='s',label='Outliers')
plt.plot(line_X,line_y_ransac,color='black',lw=2)
plt.xlabel('Average Number of Rooms [RM]')
plt.ylabel("Price in $1000s [MEDV]")
plt.legend(loc='upper left')
plt.savefig('./fig1.png')
plt.show()

结果

Step 1: 设置residual_threshold=5.0时，Inlier和Outlier，及其传统Linear Regression模型和RANSAC模型的误差，分别如下：
在这里插入图片描述
均方根误差和R2_Score分别如下：

MSE: 43.601, R2_Score:  0.484 Via Linear Regression Model
MSE: 45.620, R2_Score:  0.460 Via RANSAC

Step 2: 设置residual_threshold=1.0时，Inlier和Outlier，及其传统Linear Regression模型和RANSAC模型的误差，分别如下：
在这里插入图片描述
均方根误差和R2_Score分别如下：

MSE: 43.601, R2_Score:  0.484 Via Linear Regression Model
MSE: 44.256, R2_Score:  0.476 Via RANSAC

由上图可以得知，基于RANSAC算法构建的线性回归模型，是构建于符合预限误差内的正常点，而非过于异常的异常点。

参考文献
Sebastian Raschka, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.

来源：CSDN

作者：Santorinisu

链接：https://blog.csdn.net/Santorinisu/article/details/104457375

标签

ransac

random

mse