使用Python实现高斯朴素贝叶斯：全面指南

1. 高斯朴素贝叶斯简介

高斯朴素贝叶斯 (GNB) 是一种基于贝叶斯定理的概率分类算法，假设特征服从正态分布。它对于连续数据特别有效，实现简单且计算需求相对较低。尽管其假设过于简化，GNB 在文本分类和医学诊断等任务中通常表现得非常优秀。

高斯朴素贝叶斯的关键特点：

概率模型：为预测提供概率。
特征独立性假设：通过假设特征独立简化计算。
效率高：训练和预测阶段速度快。

2. 了解数据集

在我们的实现中，将使用两个数据集：

鸢尾花数据集：机器学习中的经典数据集，包括来自三个不同种类（Setosa，Virginica 和 Versicolor）的150个鸢尾花样本。每个样本有四个特征：花萼长度、花萼宽度、花瓣长度和花瓣宽度。
WeatherAUS 数据集：来自 Kaggle，该数据集包含来自澳大利亚气象站的气象数据，包括温度、降雨量、湿度和风速等特征。

3. 数据预处理

有效的数据预处理对于构建稳健的机器学习模型至关重要。我们将详细介绍应用于 WeatherAUS 数据集的基本预处理步骤。

a. 处理缺失数据

缺失数据会扭曲分析结果。我们采用两种策略来处理缺失值：

数值特征：使用均值策略进行填补。
分类特征：使用最频繁策略进行填补。

import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer

# Load the dataset
data = pd.read_csv('weatherAUS.csv')

# Separate features and target
X = data.iloc[:, :-1]
y = data.iloc[:, -1]

# Identify numerical and categorical columns
numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns
categorical_cols = X.select_dtypes(include=['object']).columns

# Impute numerical features with mean
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')
X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])

# Impute categorical features with the most frequent value
imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
X[categorical_cols] = imp_freq.fit_transform(X[categorical_cols])

import numpy as np

import pandas as pd

from sklearn.impute import SimpleImputer

# Load the dataset

data = pd.read_csv('weatherAUS.csv')

# Separate features and target

X = data.iloc[:, :-1]

y = data.iloc[:, -1]

# Identify numerical and categorical columns

numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns

categorical_cols = X.select_dtypes(include=['object']).columns

# Impute numerical features with mean

imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')

X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])

# Impute categorical features with the most frequent value

imp_freq = SimpleImputer(missing_values=np.nan, strategy='most_frequent')

X[categorical_cols] = imp_freq.fit_transform(X[categorical_cols])

b. 编码分类变量

机器学习算法需要数值输入。我们应用 标签编码 和 独热编码 来转换分类变量。

from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.compose import ColumnTransformer

# Label Encoding for binary categorical variables
le = LabelEncoder()
y = le.fit_transform(y)

# Function for encoding
def EncodingSelection(X, threshold=10):
    string_cols = list(X.select_dtypes(include=['object']).columns)
    one_hot_encoding_cols = []

    for col in string_cols:
        unique_vals = len(X[col].unique())
        if unique_vals == 2 or unique_vals &gt; threshold:
            X[col] = le.fit_transform(X[col])
        else:
            one_hot_encoding_cols.append(col)
    
    # One-Hot Encoding for remaining categorical variables
    if one_hot_encoding_cols:
        ct = ColumnTransformer([('encoder', OneHotEncoder(), one_hot_encoding_cols)], remainder='passthrough')
        X = ct.fit_transform(X)
    
    return X

X = EncodingSelection(X)

from sklearn.preprocessing import LabelEncoder, OneHotEncoder

from sklearn.compose import ColumnTransformer

# Label Encoding for binary categorical variables

le = LabelEncoder()

y = le.fit_transform(y)

# Function for encoding

def EncodingSelection(X, threshold=10):

string_cols = list(X.select_dtypes(include=['object']).columns)

one_hot_encoding_cols = []

for col in string_cols:

unique_vals = len(X[col].unique())

if unique_vals == 2 or unique_vals > threshold:

X[col] = le.fit_transform(X[col])

else:

one_hot_encoding_cols.append(col)

# One-Hot Encoding for remaining categorical variables

if one_hot_encoding_cols:

ct = ColumnTransformer([('encoder', OneHotEncoder(), one_hot_encoding_cols)], remainder='passthrough')

X = ct.fit_transform(X)

return X

X = EncodingSelection(X)

c. 特征选择

为了提高模型性能并减少计算成本，我们使用 SelectKBest 方法结合 卡方检验 评分函数选择最相关的特征。

from sklearn.feature_selection import SelectKBest, chi2
from sklearn.preprocessing import MinMaxScaler

# Scale features
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Select top 2 features
kbest = SelectKBest(score_func=chi2, k=2)
X_selected = kbest.fit_transform(X_scaled, y)

print(f"Selected Features Shape: {X_selected.shape}")

from sklearn.feature_selection import SelectKBest, chi2

from sklearn.preprocessing import MinMaxScaler

# Scale features

scaler = MinMaxScaler()

X_scaled = scaler.fit_transform(X)

# Select top 2 features

kbest = SelectKBest(score_func=chi2, k=2)

X_selected = kbest.fit_transform(X_scaled, y)

print(f"Selected Features Shape: {X_selected.shape}")

d. 特征缩放

标准化特征确保每个特征对结果的贡献相等，这对于基于距离的算法如KNN尤其重要。

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler(with_mean=False)
X_scaled = scaler.fit_transform(X_selected)

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler(with_mean=False)

X_scaled = scaler.fit_transform(X_selected)

4. 模型实现

我们将实现三种分类模型：K-近邻算法 (KNN)，逻辑回归 和 高斯朴素贝叶斯。

a. K-近邻算法 (KNN)

KNN基于其最近邻的多数标签对数据点进行分类。

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

# Split the dataset
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.20, random_state=1)

# Initialize and train KNN
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_train, y_train)

# Predict and evaluate
y_pred_knn = knn.predict(X_test)
knn_accuracy = accuracy_score(y_pred_knn, y_test)
print(f"KNN Accuracy: {knn_accuracy:.2f}")

from sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import accuracy_score

from sklearn.model_selection import train_test_split

# Split the dataset

X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.20, random_state=1)

# Initialize and train KNN

knn = KNeighborsClassifier(n_neighbors=3)

knn.fit(X_train, y_train)

# Predict and evaluate

y_pred_knn = knn.predict(X_test)

knn_accuracy = accuracy_score(y_pred_knn, y_test)

print(f"KNN Accuracy: {knn_accuracy:.2f}")

输出：

KNN Accuracy: 0.80

1	KNN Accuracy: 0.80

b. 逻辑回归

逻辑回归模拟分类因变量的概率。

from sklearn.linear_model import LogisticRegression

# Initialize and train Logistic Regression
lr = LogisticRegression(random_state=0, max_iter=200)
lr.fit(X_train, y_train)

# Predict and evaluate
y_pred_lr = lr.predict(X_test)
lr_accuracy = accuracy_score(y_pred_lr, y_test)
print(f"Logistic Regression Accuracy: {lr_accuracy:.2f}")

from sklearn.linear_model import LogisticRegression

# Initialize and train Logistic Regression

lr = LogisticRegression(random_state=0, max_iter=200)

lr.fit(X_train, y_train)

# Predict and evaluate

y_pred_lr = lr.predict(X_test)

lr_accuracy = accuracy_score(y_pred_lr, y_test)

print(f"Logistic Regression Accuracy: {lr_accuracy:.2f}")

输出：

Logistic Regression Accuracy: 0.83

1	Logistic Regression Accuracy: 0.83

c. 高斯朴素贝叶斯

GaussianNB 假设与每个类别相关的连续值服从正态分布。

from sklearn.naive_bayes import GaussianNB

# Initialize and train GaussianNB
gnb = GaussianNB()
gnb.fit(X_train, y_train)

# Predict and evaluate
y_pred_gnb = gnb.predict(X_test)
gnb_accuracy = accuracy_score(y_pred_gnb, y_test)
print(f"Gaussian Naive Bayes Accuracy: {gnb_accuracy:.2f}")

from sklearn.naive_bayes import GaussianNB

# Initialize and train GaussianNB

gnb = GaussianNB()

gnb.fit(X_train, y_train)

# Predict and evaluate

y_pred_gnb = gnb.predict(X_test)

gnb_accuracy = accuracy_score(y_pred_gnb, y_test)

print(f"Gaussian Naive Bayes Accuracy: {gnb_accuracy:.2f}")

输出：

Gaussian Naive Bayes Accuracy: 0.80

1	Gaussian Naive Bayes Accuracy: 0.80

5. 模型评估

模型评估对于了解模型在未见数据上的表现至关重要。我们使用 准确率 作为主要指标。

模型	准确率
K-近邻算法 (KNN)	80%
逻辑回归	83%
高斯朴素贝叶斯	80%

在测试的模型中，逻辑回归 在此数据集上优于KNN和高斯朴素贝叶斯，强调了基于数据特征进行模型选择的重要性。

6. 可视化决策边界

可视化决策边界有助于理解不同分类器如何分隔数据。我们将使用鸢尾花数据集进行此目的。

from mlxtend.plotting import plot_decision_regions
import matplotlib.pyplot as plt
from sklearn import datasets

def visualize_decision_regions(X, y, model):
    plot_decision_regions(X, y, clf=model)
    plt.xlabel('Feature 1')
    plt.ylabel('Feature 2')
    plt.title(f'Decision Boundary for {model.__class__.__name__}')
    plt.show()

# Load Iris dataset
iris = datasets.load_iris()
X_iris = iris.data[:, :2]  # First two features
y_iris = iris.target

# Initialize classifiers
knn_iris = KNeighborsClassifier(n_neighbors=3)
knn_iris.fit(X_iris, y_iris)

lr_iris = LogisticRegression(random_state=0, max_iter=200)
lr_iris.fit(X_iris, y_iris)

gnb_iris = GaussianNB()
gnb_iris.fit(X_iris, y_iris)

# Visualize decision boundaries
visualize_decision_regions(X_iris, y_iris, knn_iris)
visualize_decision_regions(X_iris, y_iris, lr_iris)
visualize_decision_regions(X_iris, y_iris, gnb_iris)

from mlxtend.plotting import plot_decision_regions

import matplotlib.pyplot as plt

from sklearn import datasets

def visualize_decision_regions(X, y, model):

plot_decision_regions(X, y, clf=model)

plt.xlabel('Feature 1')

plt.ylabel('Feature 2')

plt.title(f'Decision Boundary for {model.__class__.__name__}')

plt.show()

# Load Iris dataset

iris = datasets.load_iris()

X_iris = iris.data[:, :2] # First two features

y_iris = iris.target

# Initialize classifiers

knn_iris = KNeighborsClassifier(n_neighbors=3)

knn_iris.fit(X_iris, y_iris)

lr_iris = LogisticRegression(random_state=0, max_iter=200)

lr_iris.fit(X_iris, y_iris)

gnb_iris = GaussianNB()

gnb_iris.fit(X_iris, y_iris)

# Visualize decision boundaries

visualize_decision_regions(X_iris, y_iris, knn_iris)

visualize_decision_regions(X_iris, y_iris, lr_iris)

visualize_decision_regions(X_iris, y_iris, gnb_iris)

可视化：

K-近邻算法 (KNN)：基于邻近度捕捉更复杂的边界。
逻辑回归：线性决策边界。
高斯朴素贝叶斯：由于概率假设，边界呈曲线形。

7. 超参数调优

虽然我们的初步实验提供了良好的起点，但微调超参数可以进一步提高模型性能。可以采用 网格搜索 和 随机搜索 等技术，为每个分类器找到最佳的超参数组合。

from sklearn.model_selection import GridSearchCV

# Example: Hyperparameter tuning for KNN
param_grid = {'n_neighbors': range(1, 10)}
grid_knn = GridSearchCV(KNeighborsClassifier(), param_grid, cv=5)
grid_knn.fit(X_train, y_train)

print(f"Best KNN Parameters: {grid_knn.best_params_}")
print(f"Best KNN Accuracy: {grid_knn.best_score_:.2f}")

from sklearn.model_selection import GridSearchCV

# Example: Hyperparameter tuning for KNN

param_grid = {'n_neighbors': range(1, 10)}

grid_knn = GridSearchCV(KNeighborsClassifier(), param_grid, cv=5)

grid_knn.fit(X_train, y_train)

print(f"Best KNN Parameters: {grid_knn.best_params_}")

print(f"Best KNN Accuracy: {grid_knn.best_score_:.2f}")

8. 结论

在Python中实现高斯朴素贝叶斯非常简单，得益于像 scikit-learn 这样的库。尽管其简单性，GNB 提供了具有竞争力的性能，使其成为机器学习工具库中的宝贵工具。然而，如前所述，模型性能取决于数据集的性质。例如，在我们使用WeatherAUS数据集的实验中，逻辑回归优于GNB和KNN。

主要收获：

数据预处理：处理缺失数据和编码分类变量是关键步骤。
特征选择：选择相关特征可以提升模型性能并减少计算开销。
模型选择：始终尝试多种模型以确定哪个在特定数据集上表现最佳。
可视化：理解决策边界有助于深入了解模型如何分隔数据。

通过遵循本指南中概述的步骤，您可以有效地实现并评估高斯朴素贝叶斯以及其他分类算法，从而在您的机器学习项目中做出明智的决策。

S21L06 – 基于 Python 的高斯朴素贝叶斯

使用Python实现高斯朴素贝叶斯：全面指南

目录

1. 高斯朴素贝叶斯简介

2. 了解数据集

3. 数据预处理

a. 处理缺失数据

b. 编码分类变量

c. 特征选择

d. 特征缩放

4. 模型实现

a. K-近邻算法 (KNN)

b. 逻辑回归

c. 高斯朴素贝叶斯

5. 模型评估

6. 可视化决策边界

7. 超参数调优

8. 结论

9. 参考文献