非线性方程的拟合，例如 y=x^2+0.5x+1 , 就是将x^2看作X的一个特征值

#准备数据
import numpy as np
import matplotlib.pyplot as pltx = np.random.uniform(-3,3,size=100)
X = x.reshape(-1,1)y = 0.5*x**2 + x + 2 +np.random.normal(0,1,size=100)

X2 = np.hstack([X,X**2])  #将X和X^2共同作为特征值构成心得矩阵from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X2,y)plt.scatter(x,y)
plt.plot(np.sort(x),lin_reg.predict(X2)[np.argsort(x)],color='r')

 

 

scikit-learn中的多项式回归

from sklearn.preprocessing import PolynomialFeaturespoly = PolynomialFeatures(degree=3)
poly.fit(X)
X2 = poly.transform(X)
# X2.shape = (100,4)

将X拓展为了具有X，X^2,X^3的矩阵

当数据具有多个特征时，即X不止有一列时

 

 

多项式回归应用于PipeLine

---

使用PipeLine管道一次性进行多项式回归、数据归一化、线性回归预测

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScalerpoly_reg = Pipeline([("poly",PolynomialFeatures(degree=2)),("std_scaler",StandardScaler()),("lin_reg",LinearRegression())
])poly_reg.fit(X,y)y_predict = poly_reg.predict(X)plt.scatter(x,y)
plt.plot(np.sort(x),y_predict[np.argsort(x)],color='r')
plt.show()

 

学习曲线

---

由学习曲线看欠拟合和过拟合，横轴代表训练用数据数量，纵轴为均方根误差

def plot_learning_curve(algo,X_train,X_test,y_train,y_test):train_score = []test_score = []for i in range(1,len(X_train)+1):algo.fit(X_train[:i],y_train[:i])y_train_predict = algo.predict(X_train[:i])train_score.append(sqrt(mean_squared_error(y_train_predict[:i],y_train[:i])))y_test_predict = algo.predict(X_test)test_score.append(sqrt(mean_squared_error(y_test_predict,y_test)))plt.plot([i for i in range(1,len(X_train)+1)],train_score,label="train")plt.plot([i for i in range(1,len(X_train)+1)],test_score,label="test")plt.legend()plt.axis([0,len(X_train)+1,0,4])plt.show()

plot_learning_curve(LinearRegression(),X_train,X_test,y_train,y_test)

欠拟合，误差较大

poly2_reg = PolynomialRegression(degree=2)
plot_learning_curve(poly2_reg,X_train,X_test,y_train,y_test)

最佳

poly20_reg = PolynomialRegression(degree=20)
plot_learning_curve(poly20_reg,X_train,X_test,y_train,y_test)

过拟合，泛化能力较差，较好拟合训练集，难以拟合测试集