在Python中实现决策树、随机森林、XGBoost和AdaBoost进行天气预测

简介

预测天气状况是机器学习中的经典问题，为农业、航空和活动策划等多个行业提供了宝贵的见解。在本综合指南中，我们将深入探讨实现几种机器学习模型——包括决策树、随机森林、XGBoost和AdaBoost——以预测明天是否会下雨，使用的是澳大利亚天气数据集。我们将详细介绍数据预处理、模型训练、评估，甚至将这些模型部署到实际的网络应用中。

数据集概述

澳大利亚天气数据集，来源于Kaggle，包含与澳大利亚各地记录的天气状况相关的24个特征。主要目标是预测RainTomorrow属性，指示第二天是否会下雨。

数据集特征

Date: 观测日期。
Location: 气象站的地理位置。
MinTemp: 最低温度，单位°C。
MaxTemp: 最高温度，单位°C。
Rainfall: 降雨量，单位毫米。
Evaporation: 蒸发量，单位毫米。
Sunshine: 日照时数。
WindGustDir: 最强阵风的风向。
WindGustSpeed: 最强阵风的风速，单位km/h。
WindDir9am: 上午9点的风向。
WindDir3pm: 下午3点的风向。
…以及更多。

数据预处理

有效的数据预处理对构建准确可靠的机器学习模型至关重要。我们将涵盖处理缺失值、编码分类变量、特征选择和缩放。

处理缺失值

缺失数据可能会显著影响模型性能。我们将分别处理数值和分类数据的缺失值。

数值数据

对于数值列，我们将使用均值填充来填补缺失值。

import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer

# Load data
data = pd.read_csv('weatherAUS.csv')

# Identify numerical columns
numerical_cols = data.select_dtypes(include=['int64', 'float64']).columns

# Impute missing values with mean
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')
data[numerical_cols] = imp_mean.fit_transform(data[numerical_cols])

import numpy as np

import pandas as pd

from sklearn.impute import SimpleImputer

# Load data

data = pd.read_csv('weatherAUS.csv')

# Identify numerical columns

numerical_cols = data.select_dtypes(include=['int64', 'float64']).columns

# Impute missing values with mean

imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')

data[numerical_cols] = imp_mean.fit_transform(data[numerical_cols])

分类数据

对于分类列，我们将使用众数填充。

# Identify categorical columns
categorical_cols = data.select_dtypes(include=['object']).columns

# Impute missing values with the most frequent value
imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
data[categorical_cols] = imp_freq.fit_transform(data[categorical_cols])

# Identify categorical columns

categorical_cols = data.select_dtypes(include=['object']).columns

# Impute missing values with the most frequent value

imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')

data[categorical_cols] = imp_freq.fit_transform(data[categorical_cols])

编码分类变量

机器学习算法需要数值输入。我们将根据每个特征中唯一类别的数量，采用标签编码和独热编码。

from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.compose import ColumnTransformer

def encode_features(df, threshold=10):
    label_enc_cols = [col for col in df.columns if df[col].dtype == 'object' and df[col].nunique() &lt;= threshold]
    onehot_enc_cols = [col for col in df.columns if df[col].dtype == 'object' and df[col].nunique() &gt; threshold]
    
    # Label Encoding
    le = LabelEncoder()
    for col in label_enc_cols:
        df[col] = le.fit_transform(df[col])
    
    # One-Hot Encoding
    ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), onehot_enc_cols)], remainder='passthrough')
    df = ct.fit_transform(df)
    
    return df

X = data.drop('RainTomorrow', axis=1)
y = data['RainTomorrow']

X = encode_features(X)

from sklearn.preprocessing import LabelEncoder, OneHotEncoder

from sklearn.compose import ColumnTransformer

def encode_features(df, threshold=10):

label_enc_cols = [col for col in df.columns if df[col].dtype == 'object' and df[col].nunique() <= threshold]

onehot_enc_cols = [col for col in df.columns if df[col].dtype == 'object' and df[col].nunique() > threshold]

# Label Encoding

le = LabelEncoder()

for col in label_enc_cols:

df[col] = le.fit_transform(df[col])

# One-Hot Encoding

ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), onehot_enc_cols)], remainder='passthrough')

df = ct.fit_transform(df)

return df

X = data.drop('RainTomorrow', axis=1)

y = data['RainTomorrow']

X = encode_features(X)

特征选择

为了提高模型性能并减少计算复杂性，我们将使用SelectKBest方法与卡方统计量选择前几个重要特征。

from sklearn.feature_selection import SelectKBest, chi2
from sklearn.preprocessing import MinMaxScaler

# Scale features
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Select top 10 features
selector = SelectKBest(score_func=chi2, k=10)
X_selected = selector.fit_transform(X_scaled, y)

# Further reduce to top 2 features for visualization
best_features = selector.get_support(indices=True)
X_final = X_selected[:, :2]

from sklearn.feature_selection import SelectKBest, chi2

from sklearn.preprocessing import MinMaxScaler

# Scale features

scaler = MinMaxScaler()

X_scaled = scaler.fit_transform(X)

# Select top 10 features

selector = SelectKBest(score_func=chi2, k=10)

X_selected = selector.fit_transform(X_scaled, y)

# Further reduce to top 2 features for visualization

best_features = selector.get_support(indices=True)

X_final = X_selected[:, :2]

训练集与测试集划分及特征缩放

将数据划分为训练集和测试集，确保模型性能在未见过的数据上得到评估。

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Split the data
X_train, X_test, y_train, y_test = train_test_split(X_final, y, test_size=0.20, random_state=1)

# Feature Scaling
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

# Split the data

X_train, X_test, y_train, y_test = train_test_split(X_final, y, test_size=0.20, random_state=1)

# Feature Scaling

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

模型实现与评估

我们将实现各种机器学习模型，并使用准确率来评估它们的性能。

K-近邻算法 (KNN)

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_train, y_train)
y_pred_knn = knn.predict(X_test)
knn_accuracy = accuracy_score(y_pred_knn, y_test)
print(f'KNN Accuracy: {knn_accuracy:.2f}')

from sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import accuracy_score

knn = KNeighborsClassifier(n_neighbors=3)

knn.fit(X_train, y_train)

y_pred_knn = knn.predict(X_test)

knn_accuracy = accuracy_score(y_pred_knn, y_test)

print(f'KNN Accuracy: {knn_accuracy:.2f}')

KNN 准确率: 0.80

逻辑回归

from sklearn.linear_model import LogisticRegression

log_reg = LogisticRegression(random_state=0, max_iter=200)
log_reg.fit(X_train, y_train)
y_pred_lr = log_reg.predict(X_test)
lr_accuracy = accuracy_score(y_pred_lr, y_test)
print(f'Logistic Regression Accuracy: {lr_accuracy:.2f}')

from sklearn.linear_model import LogisticRegression

log_reg = LogisticRegression(random_state=0, max_iter=200)

log_reg.fit(X_train, y_train)

y_pred_lr = log_reg.predict(X_test)

lr_accuracy = accuracy_score(y_pred_lr, y_test)

print(f'Logistic Regression Accuracy: {lr_accuracy:.2f}')

逻辑回归准确率: 0.83

高斯朴素贝叶斯

from sklearn.naive_bayes import GaussianNB

gnb = GaussianNB()
gnb.fit(X_train, y_train)
y_pred_gnb = gnb.predict(X_test)
gnb_accuracy = accuracy_score(y_pred_gnb, y_test)
print(f'Gaussian Naive Bayes Accuracy: {gnb_accuracy:.2f}')

from sklearn.naive_bayes import GaussianNB

gnb = GaussianNB()

gnb.fit(X_train, y_train)

y_pred_gnb = gnb.predict(X_test)

gnb_accuracy = accuracy_score(y_pred_gnb, y_test)

print(f'Gaussian Naive Bayes Accuracy: {gnb_accuracy:.2f}')

高斯朴素贝叶斯准确率: 0.80

支持向量机 (SVM)

from sklearn.svm import SVC

svm = SVC()
svm.fit(X_train, y_train)
y_pred_svm = svm.predict(X_test)
svm_accuracy = accuracy_score(y_pred_svm, y_test)
print(f'SVM Accuracy: {svm_accuracy:.2f}')

from sklearn.svm import SVC

svm = SVC()

svm.fit(X_train, y_train)

y_pred_svm = svm.predict(X_test)

svm_accuracy = accuracy_score(y_pred_svm, y_test)

print(f'SVM Accuracy: {svm_accuracy:.2f}')

SVM 准确率: 0.83

决策树

from sklearn.tree import DecisionTreeClassifier

dtc = DecisionTreeClassifier()
dtc.fit(X_train, y_train)
y_pred_dtc = dtc.predict(X_test)
dtc_accuracy = accuracy_score(y_pred_dtc, y_test)
print(f'Decision Tree Accuracy: {dtc_accuracy:.2f}')

from sklearn.tree import DecisionTreeClassifier

dtc = DecisionTreeClassifier()

dtc.fit(X_train, y_train)

y_pred_dtc = dtc.predict(X_test)

dtc_accuracy = accuracy_score(y_pred_dtc, y_test)

print(f'Decision Tree Accuracy: {dtc_accuracy:.2f}')

决策树准确率: 0.83

随机森林

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=500, max_depth=5)
rf.fit(X_train, y_train)
y_pred_rf = rf.predict(X_test)
rf_accuracy = accuracy_score(y_pred_rf, y_test)
print(f'Random Forest Accuracy: {rf_accuracy:.2f}')

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=500, max_depth=5)

rf.fit(X_train, y_train)

y_pred_rf = rf.predict(X_test)

rf_accuracy = accuracy_score(y_pred_rf, y_test)

print(f'Random Forest Accuracy: {rf_accuracy:.2f}')

随机森林准确率: 0.83

XGBoost和AdaBoost

虽然初始实现未涵盖XGBoost和AdaBoost，这些集成方法可以进一步提升模型性能。以下是如何实现它们的简要示例：

XGBoost

from xgboost import XGBClassifier

xgb = XGBClassifier(use_label_encoder=False, eval_metric='logloss')
xgb.fit(X_train, y_train)
y_pred_xgb = xgb.predict(X_test)
xgb_accuracy = accuracy_score(y_pred_xgb, y_test)
print(f'XGBoost Accuracy: {xgb_accuracy:.2f}')

from xgboost import XGBClassifier

xgb = XGBClassifier(use_label_encoder=False, eval_metric='logloss')

xgb.fit(X_train, y_train)

y_pred_xgb = xgb.predict(X_test)

xgb_accuracy = accuracy_score(y_pred_xgb, y_test)

print(f'XGBoost Accuracy: {xgb_accuracy:.2f}')

AdaBoost

from sklearn.ensemble import AdaBoostClassifier

ada = AdaBoostClassifier(n_estimators=100, random_state=0)
ada.fit(X_train, y_train)
y_pred_ada = ada.predict(X_test)
ada_accuracy = accuracy_score(y_pred_ada, y_test)
print(f'AdaBoost Accuracy: {ada_accuracy:.2f}')

from sklearn.ensemble import AdaBoostClassifier

ada = AdaBoostClassifier(n_estimators=100, random_state=0)

ada.fit(X_train, y_train)

y_pred_ada = ada.predict(X_test)

ada_accuracy = accuracy_score(y_pred_ada, y_test)

print(f'AdaBoost Accuracy: {ada_accuracy:.2f}')

注意: 确保您已使用pip install xgboost安装了xgboost库。

可视化决策区域

可视化决策边界有助于理解不同模型如何分类数据。以下是使用鸢尾花数据集的示例：

from mlxtend.plotting import plot_decision_regions
import matplotlib.pyplot as plt
from sklearn import datasets

# Load Iris dataset
iris = datasets.load_iris()
X_vis = iris.data[:, :2]
y_vis = iris.target

# Train KNN
knn_vis = KNeighborsClassifier(n_neighbors=3)
knn_vis.fit(X_vis, y_vis)

# Plot decision regions
plot_decision_regions(X_vis, y_vis, clf=knn_vis)

plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('KNN Decision Regions')
plt.legend()
plt.show()

from mlxtend.plotting import plot_decision_regions

import matplotlib.pyplot as plt

from sklearn import datasets

# Load Iris dataset

iris = datasets.load_iris()

X_vis = iris.data[:, :2]

y_vis = iris.target

# Train KNN

knn_vis = KNeighborsClassifier(n_neighbors=3)

knn_vis.fit(X_vis, y_vis)

# Plot decision regions

plot_decision_regions(X_vis, y_vis, clf=knn_vis)

plt.xlabel('Feature 1')

plt.ylabel('Feature 2')

plt.title('KNN Decision Regions')

plt.legend()

plt.show()

可视化输出: 展示KNN分类器创建的决策边界的图表。

结论

在本指南中，我们探讨了使用澳大利亚天气数据集实现各种机器学习模型——决策树、随机森林、逻辑回归、KNN、高斯朴素贝叶斯和SVM——来预测天气状况。每个模型都展示了具有竞争力的准确率，逻辑回归、SVM、决策树和随机森林的准确率均达到约83%。

为了提升性能，可以整合像XGBoost和AdaBoost这样的集成方法。此外，将这些模型部署到网络应用中，可以提供实时天气预测，使最终用户能够将这些见解付诸实践。

S24L01 – 决策树与随机森林