S17L04 – 不使用 GridSearchCV 的 K 折交叉验证（续）

html
在无需GridSearchCV的情况下实现K折交叉验证进行汽车价格预测

目录

    介绍
    数据集概述
    
        数据预处理
        
            处理缺失数据
            特征选择
        
    
    
        特征工程
        
            编码分类变量
            特征缩放
        
    
    
        构建回归模型
        
            决策树回归器
            随机森林回归器
            AdaBoost 回归器
            XGBoost 回归器
            支持向量回归器 (SVR)
        
    
    实现K折交叉验证
    评估模型性能
    结论


介绍
预测汽车价格是一个经典的回归问题，涉及根据发动机大小、马力、燃料类型等各种特征预测车辆的价格。实施K折交叉验证通过确保模型对未见数据的良好泛化性，增强了模型的可靠性。本文展示了如何在Python中进行数据预处理、特征工程、构建多个回归模型，并使用K折交叉验证评估其性能。

数据集概述
我们将使用Kaggle的Car Price Prediction数据集，该数据集包含不同车型的详细规格及其价格。数据集包括诸如symboling、CarName、fueltype、aspiration、doornumber、carbody等多个影响汽车价格的特征。

数据预处理
有效的数据预处理对于为建模准备数据集至关重要。这包括处理缺失值、编码分类变量以及选择相关特征。

处理缺失数据

数值数据
可以使用统计量来处理数值特征中的缺失值。我们将使用均值策略来填补数值列中的缺失值。




		
		
			
			
Java
			
			import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer

# Load the dataset
data = pd.read_csv('CarPrice.csv')
X = data.iloc[:, :-1]
y = data.iloc[:, -1]

# Identify numerical columns
numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns

# Impute missing values with mean
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')
X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
				
						import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer
 
# Load the dataset
data = pd.read_csv('CarPrice.csv')
X = data.iloc[:, :-1]
y = data.iloc[:, -1]
 
# Identify numerical columns
numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns
 
# Impute missing values with mean
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')
X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])
					
				
			
		



分类数据
对于分类特征，使用最频繁值策略来填补缺失值是有效的。




		
		
			
			
Java
			
			from sklearn.impute import SimpleImputer

# Identify categorical columns
string_cols = X.select_dtypes(include=['object']).columns

# Impute missing values with the most frequent value
imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
X[string_cols] = imp_freq.fit_transform(X[string_cols])
			
				
					
				
					1
2
3
4
5
6
7
8
				
						from sklearn.impute import SimpleImputer
 
# Identify categorical columns
string_cols = X.select_dtypes(include=['object']).columns
 
# Impute missing values with the most frequent value
imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
X[string_cols] = imp_freq.fit_transform(X[string_cols])
					
				
			
		



特征选择
选择相关特征有助于减少模型的复杂性并提高其性能。




		
		
			
			
Java
			
			# Drop the 'car_ID' column as it does not contribute to price prediction
X.drop('car_ID', axis=1, inplace=True)
			
				
					
				
					1
2
				
						# Drop the 'car_ID' column as it does not contribute to price prediction
X.drop('car_ID', axis=1, inplace=True)
					
				
			
		



特征工程
特征工程涉及将原始数据转化为对预测模型更有意义的特征，从而更好地表示潜在问题。

编码分类变量
机器学习算法需要数值输入，因此需要对分类变量进行编码。我们将使用独热编码将分类变量转换为二进制矩阵。




		
		
			
			
Java
			
			from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

# Identify categorical columns by their indices
string_cols = list(np.where((X.dtypes == object))[0])

# Apply One-Hot Encoding
columnTransformer = ColumnTransformer(
    [('encoder', OneHotEncoder(), string_cols)],
    remainder='passthrough'
)
X = columnTransformer.fit_transform(X)
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
				
						from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
 
# Identify categorical columns by their indices
string_cols = list(np.where((X.dtypes == object))[0])
 
# Apply One-Hot Encoding
columnTransformer = ColumnTransformer(
    [('encoder', OneHotEncoder(), string_cols)],
    remainder='passthrough'
)
X = columnTransformer.fit_transform(X)
					
				
			
		



特征缩放
缩放确保每个特征在结果中均等贡献，增强某些算法的性能。




		
		
			
			
Java
			
			from sklearn.preprocessing import StandardScaler

# Initialize the StandardScaler
sc = StandardScaler(with_mean=False)

# Fit and transform the training data
sc.fit(X_train)
X_train = sc.transform(X_train)
X_test = sc.transform(X_test)
			
				
					
				
					1
2
3
4
5
6
7
8
9
				
						from sklearn.preprocessing import StandardScaler
 
# Initialize the StandardScaler
sc = StandardScaler(with_mean=False)
 
# Fit and transform the training data
sc.fit(X_train)
X_train = sc.transform(X_train)
X_test = sc.transform(X_test)
					
				
			
		



构建回归模型
我们将构建并评估五种不同的回归模型来预测汽车价格：

    决策树回归器
    随机森林回归器
    AdaBoost 回归器
    XGBoost 回归器
    支持向量回归器 (SVR)


决策树回归器
决策树回归器根据特征值将数据分割成子集，易于解释。




		
		
			
			
Java
			
			from sklearn.tree import DecisionTreeRegressor

# Initialize the model
decisionTreeRegressor = DecisionTreeRegressor(max_depth=4)
			
				
					
				
					1
2
3
4
				
						from sklearn.tree import DecisionTreeRegressor
 
# Initialize the model
decisionTreeRegressor = DecisionTreeRegressor(max_depth=4)
					
				
			
		



随机森林回归器
随机森林通过聚合多个决策树的预测，减少过拟合并提高准确性。




		
		
			
			
Java
			
			from sklearn.ensemble import RandomForestRegressor

# Initialize the model
randomForestRegressor = RandomForestRegressor(n_estimators=25, random_state=10)
			
				
					
				
					1
2
3
4
				
						from sklearn.ensemble import RandomForestRegressor
 
# Initialize the model
randomForestRegressor = RandomForestRegressor(n_estimators=25, random_state=10)
					
				
			
		



AdaBoost 回归器
AdaBoost将多个弱学习器组合起来创建一个强大的预测模型，专注于以前预测错误的实例。




		
		
			
			
Java
			
			from sklearn.ensemble import AdaBoostRegressor

# Initialize the model
adaBoostRegressor = AdaBoostRegressor(random_state=0, n_estimators=100)
			
				
					
				
					1
2
3
4
				
						from sklearn.ensemble import AdaBoostRegressor
 
# Initialize the model
adaBoostRegressor = AdaBoostRegressor(random_state=0, n_estimators=100)
					
				
			
		



XGBoost 回归器
XGBoost是一种针对性能和速度优化的分布式梯度提升库。




		
		
			
			
Java
			
			import xgboost as xgb

# Initialize the model
xgbRegressor = xgb.XGBRegressor(
    n_estimators=100,
    reg_lambda=1,
    gamma=0,
    max_depth=3,
    learning_rate=0.05
)
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
				
						import xgboost as xgb
 
# Initialize the model
xgbRegressor = xgb.XGBRegressor(
    n_estimators=100,
    reg_lambda=1,
    gamma=0,
    max_depth=3,
    learning_rate=0.05
)
					
				
			
		



支持向量回归器 (SVR)
SVR利用支持向量机的原理进行回归任务，在高维空间中表现有效。




		
		
			
			
Java
			
			from sklearn.svm import SVR

# Initialize the model
svr = SVR()
			
				
					
				
					1
2
3
4
				
						from sklearn.svm import SVR
 
# Initialize the model
svr = SVR()
					
				
			
		



实现K折交叉验证
K折交叉验证将数据集划分为k个子集，并迭代地训练和验证模型k次，每次使用不同的子集作为验证集。




		
		
			
			
Java
			
			from sklearn.model_selection import KFold
from sklearn.metrics import r2_score

# Define the K-Fold Cross Validator
kf = KFold(n_splits=10, random_state=42, shuffle=True)

# Function to build and evaluate the model
def build_model(X_train, X_test, y_train, y_test, model):
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    return r2_score(y_test, y_pred)
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
				
						from sklearn.model_selection import KFold
from sklearn.metrics import r2_score
 
# Define the K-Fold Cross Validator
kf = KFold(n_splits=10, random_state=42, shuffle=True)
 
# Function to build and evaluate the model
def build_model(X_train, X_test, y_train, y_test, model):
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    return r2_score(y_test, y_pred)
					
				
			
		



运行K折交叉验证
我们将在K折中评估每个模型的性能，并计算平均R²分数。




		
		
			
			
Java
			
			# Initialize score lists
decisionTreeRegressor_scores = []
randomForestRegressor_scores = []
adaBoostRegressor_scores = []
xgbRegressor_scores = []
svr_scores = []

# Perform K-Fold Cross-Validation
for train_index, test_index in kf.split(X):
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    
    # Decision Tree
    decisionTreeRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, decisionTreeRegressor)
    )
    
    # Random Forest
    randomForestRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, randomForestRegressor)
    )
    
    # AdaBoost
    adaBoostRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, adaBoostRegressor)
    )
    
    # XGBoost
    xgbRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, xgbRegressor)
    )
    
    # SVR
    svr_scores.append(
        build_model(X_train, X_test, y_train, y_test, svr)
    )
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
				
						# Initialize score lists
decisionTreeRegressor_scores = []
randomForestRegressor_scores = []
adaBoostRegressor_scores = []
xgbRegressor_scores = []
svr_scores = []
 
# Perform K-Fold Cross-Validation
for train_index, test_index in kf.split(X):
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    
    # Decision Tree
    decisionTreeRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, decisionTreeRegressor)
    )
    
    # Random Forest
    randomForestRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, randomForestRegressor)
    )
    
    # AdaBoost
    adaBoostRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, adaBoostRegressor)
    )
    
    # XGBoost
    xgbRegressor_scores.append(
        build_model(X_train, X_test, y_train, y_test, xgbRegressor)
    )
    
    # SVR
    svr_scores.append(
        build_model(X_train, X_test, y_train, y_test, svr)
    )
					
				
			
		



评估模型性能
在运行K折交叉验证后，我们将计算每个模型的平均R²分数以评估其性能。




		
		
			
			
Java
			
			from statistics import mean

print('Decision Tree Regressor Mean R² Score: ', mean(decisionTreeRegressor_scores))
print('Random Forest Regressor Mean R² Score: ', mean(randomForestRegressor_scores))
print('AdaBoost Regressor Mean R² Score: ', mean(adaBoostRegressor_scores))
print('XGBoost Regressor Mean R² Score: ', mean(xgbRegressor_scores))
print('SVR Mean R² Score: ', mean(svr_scores))
			
				
					
				
					1
2
3
4
5
6
7
				
						from statistics import mean
 
print('Decision Tree Regressor Mean R² Score: ', mean(decisionTreeRegressor_scores))
print('Random Forest Regressor Mean R² Score: ', mean(randomForestRegressor_scores))
print('AdaBoost Regressor Mean R² Score: ', mean(adaBoostRegressor_scores))
print('XGBoost Regressor Mean R² Score: ', mean(xgbRegressor_scores))
print('SVR Mean R² Score: ', mean(svr_scores))
					
				
			
		


示例输出：




		
		
			
			
Java
			
			Decision Tree Regressor Mean R² Score:  0.8786768422448108
Random Forest Regressor Mean R² Score:  0.9070724684428952
AdaBoost Regressor Mean R² Score:  0.894756851083693
XGBoost Regressor Mean R² Score:  0.9049838393114154
SVR Mean R² Score:  -0.1510507928400266
			
				
					
				
					1
2
3
4
5
				
						Decision Tree Regressor Mean R² Score:  0.8786768422448108
Random Forest Regressor Mean R² Score:  0.9070724684428952
AdaBoost Regressor Mean R² Score:  0.894756851083693
XGBoost Regressor Mean R² Score:  0.9049838393114154
SVR Mean R² Score:  -0.1510507928400266
					
				
			
		


解释：

    随机森林回归器显示出最高的平均R²分数，表明在测试的模型中表现最佳。
    SVR的R²分数为负，表明在此数据集上的表现不佳，可能是由于在没有超参数调优的情况下无法有效捕捉潜在模式。


结论
实施K折交叉验证提供了一种稳健的方法来评估回归模型的性能，确保结果具有泛化性而不依赖于特定的训练-测试分割。在本指南中，我们演示了如何进行数据预处理、编码分类变量、缩放特征、构建多个回归模型，并使用无需GridSearchCV的K折交叉验证评估其性能。

关键要点：

    数据预处理：正确处理缺失数据和特征选择对模型性能至关重要。
    特征工程：编码分类变量和缩放特征可以显著影响模型学习模式的能力。
    模型评估：K折交叉验证提供了一种可靠的方法来评估模型在未见数据上的泛化能力。
    模型选择：在测试的模型中，集成方法如随机森林和XGBoost在这种特定情况下表现优于简单模型如决策树和SVR。

为了进一步优化，可以集成超参数调优技术，如GridSearchCV或RandomizedSearchCV，通过为每个算法找到最佳参数集来提升模型性能。

---

通过遵循这种结构化的方法，您可以有效地为各种回归任务实施K折交叉验证，确保您的模型既准确又稳健。