S20L03 – Python下的逻辑回归

html
在Python中实现逻辑回归：全面指南

利用Python的Scikit-Learn库释放逻辑回归的威力。学习如何预处理数据、处理缺失值、执行特征选择以及构建高效的分类模型。通过这个循序渐进的教程提升您的机器学习技能。



逻辑回归简介

逻辑回归是机器学习中的基础算法，主要用于二元分类任务。与预测连续结果的线性回归不同，逻辑回归基于一个或多个预测变量估计二元结果的概率。

在本全面指南中，我们将介绍如何使用Scikit-Learn在Python中实现逻辑回归模型。我们将涵盖数据预处理、处理缺失值、编码分类变量、特征选择、缩放以及模型评估。此外，我们还将比较逻辑回归与K-最近邻（KNN）分类器的性能。

目录

    理解逻辑回归
    设置环境
    数据探索与预处理
    处理缺失数据
    编码分类变量
    特征选择
    特征缩放
    训练模型
    评估模型性能
    超参数调优
    结论


理解逻辑回归

逻辑回归是一种用于分类任务的线性模型。它预测给定输入属于特定类的概率。输出通过逻辑函数（Sigmoid函数）转换，确保输出值位于0和1之间。

关键特性：

    二元分类：适用于目标变量具有两个类别的情形。
    概率估计：提供类别成员身份的概率。
    线性决策边界：假设输入特征与结果的对数几率之间存在线性关系。


设置环境

在开始编码之前，请确保已安装必要的库。我们将使用Pandas进行数据操作，NumPy进行数值运算，Scikit-Learn进行机器学习算法，以及Seaborn进行数据可视化。





		
		
			
			
Java
			
			import pandas as pd
import numpy as np
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, LabelEncoder, StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.feature_selection import SelectKBest, chi2
from sklearn.metrics import accuracy_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
import matplotlib.pyplot as plt
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
				
						import pandas as pd
import numpy as np
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, LabelEncoder, StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.feature_selection import SelectKBest, chi2
from sklearn.metrics import accuracy_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
import matplotlib.pyplot as plt
					
				
			
		



数据探索与预处理

在本教程中，我们将使用澳大利亚天气数据集。该数据集包含了各个澳大利亚城市的天气观测记录。

加载数据





		
		
			
			
Java
			
			# Load the dataset
data = pd.read_csv('weatherAUS.csv')
			
				
					
				
					1
2
				
						# Load the dataset
data = pd.read_csv('weatherAUS.csv')
					
				
			
		



让我们查看最后几行以了解数据结构：





		
		
			
			
Java
			
			data.tail()
			
				
					
				
					1
				
						data.tail()
					
				
			
		



示例输出：


    
        Date
        Location
        MinTemp
        MaxTemp
        Rainfall
        Evaporation
        ...
        RainToday
        RISK_MM
        RainTomorrow
    
    
        2017-06-20
        Uluru
        3.5
        21.8
        0.0
        NaN
        ...
        No
        0.0
        No
    
    
        2017-06-21
        Uluru
        2.8
        23.4
        0.0
        NaN
        ...
        No
        0.0
        No
    
    
        2017-06-22
        Uluru
        3.6
        25.3
        0.0
        NaN
        ...
        No
        0.0
        No
    
    
        2017-06-23
        Uluru
        5.4
        26.9
        0.0
        NaN
        ...
        No
        0.0
        No
    
    
        2017-06-24
        Uluru
        7.8
        27.0
        0.0
        NaN
        ...
        No
        0.0
        No
    


分离特征和目标变量





		
		
			
			
Java
			
			# Features
X = data.iloc[:, :-1]

# Target variable
y = data.iloc[:, -1]
			
				
					
				
					1
2
3
4
5
				
						# Features
X = data.iloc[:, :-1]
 
# Target variable
y = data.iloc[:, -1]
					
				
			
		



处理特定数据集需求：

如果您只使用澳大利亚天气数据集，可能需要删除特定列：





		
		
			
			
Java
			
			X.drop('RISK_MM', axis=1, inplace=True)
			
				
					
				
					1
				
						X.drop('RISK_MM', axis=1, inplace=True)
					
				
			
		



处理缺失数据

现实世界的数据集通常包含缺失值。正确处理缺失值对于确保模型准确性至关重要。

处理数值数据

我们将使用Scikit-Learn的SimpleImputer将缺失的数值值替换为每列的均值。





		
		
			
			
Java
			
			from sklearn.impute import SimpleImputer

# Identify numerical columns
numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns

# Initialize the imputer
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')

# Fit and transform the data
X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
				
						from sklearn.impute import SimpleImputer
 
# Identify numerical columns
numerical_cols = X.select_dtypes(include=['int64', 'float64']).columns
 
# Initialize the imputer
imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')
 
# Fit and transform the data
X[numerical_cols] = imp_mean.fit_transform(X[numerical_cols])
					
				
			
		



处理分类数据

对于分类变量，我们将用最频繁的类别替换缺失值。





		
		
			
			
Java
			
			# Identify string columns
string_cols = X.select_dtypes(include=['object']).columns

# Initialize the imputer
imp_mode = SimpleImputer(missing_values=np.nan, strategy='most_frequent')

# Fit and transform the data
X[string_cols] = imp_mode.fit_transform(X[string_cols])
			
				
					
				
					1
2
3
4
5
6
7
8
				
						# Identify string columns
string_cols = X.select_dtypes(include=['object']).columns
 
# Initialize the imputer
imp_mode = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
 
# Fit and transform the data
X[string_cols] = imp_mode.fit_transform(X[string_cols])
					
				
			
		



编码分类变量

机器学习模型需要数值输入。我们将根据唯一类别的数量使用独热编码和标签编码转换分类变量。

独热编码

适用于具有少量唯一类别的分类变量。





		
		
			
			
Java
			
			def OneHotEncoderMethod(indices, data):
    columnTransformer = ColumnTransformer(
        transformers=[('encoder', OneHotEncoder(), indices)],
        remainder='passthrough'
    )
    return columnTransformer.fit_transform(data)

one_hot_indices = [X.columns.get_loc(col) for col in string_cols if X[col].nunique() <= 10]
X = OneHotEncoderMethod(one_hot_indices, X)
			
				
					
				
					1
2
3
4
5
6
7
8
9
				
						def OneHotEncoderMethod(indices, data):
    columnTransformer = ColumnTransformer(
        transformers=[('encoder', OneHotEncoder(), indices)],
        remainder='passthrough'
    )
    return columnTransformer.fit_transform(data)
 
one_hot_indices = [X.columns.get_loc(col) for col in string_cols if X[col].nunique() <= 10]
X = OneHotEncoderMethod(one_hot_indices, X)
					
				
			
		



标签编码

适用于二元分类变量。





		
		
			
			
Java
			
			def LabelEncoderMethod(series):
    le = LabelEncoder()
    return le.fit_transform(series)

binary_cols = [col for col in string_cols if X[col].nunique() == 2]
for col in binary_cols:
    X[col] = LabelEncoderMethod(X[col])
			
				
					
				
					1
2
3
4
5
6
7
				
						def LabelEncoderMethod(series):
    le = LabelEncoder()
    return le.fit_transform(series)
 
binary_cols = [col for col in string_cols if X[col].nunique() == 2]
for col in binary_cols:
    X[col] = LabelEncoderMethod(X[col])
					
				
			
		



对X进行编码选择

对于具有超过两个类别（并超过某个阈值）的分类变量，我们将使用标签编码。否则，我们将应用独热编码。





		
		
			
			
Java
			
			def EncodingSelection(X, threshold=10):
    string_cols = X.select_dtypes(include=['object']).columns
    one_hot_encoding_indices = []
    
    for col in string_cols:
        unique_count = X[col].nunique()
        if unique_count == 2 or unique_count > threshold:
            X[col] = LabelEncoderMethod(X[col])
        else:
            one_hot_encoding_indices.append(X.columns.get_loc(col))
    
    X = OneHotEncoderMethod(one_hot_encoding_indices, X)
    return X

X = EncodingSelection(X)
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
				
						def EncodingSelection(X, threshold=10):
    string_cols = X.select_dtypes(include=['object']).columns
    one_hot_encoding_indices = []
    
    for col in string_cols:
        unique_count = X[col].nunique()
        if unique_count == 2 or unique_count > threshold:
            X[col] = LabelEncoderMethod(X[col])
        else:
            one_hot_encoding_indices.append(X.columns.get_loc(col))
    
    X = OneHotEncoderMethod(one_hot_encoding_indices, X)
    return X
 
X = EncodingSelection(X)
					
				
			
		



特征选择

为了提高模型性能并减少过拟合，我们将使用卡方检验选择前几个特征。





		
		
			
			
Java
			
			from sklearn.feature_selection import SelectKBest, chi2
from sklearn.preprocessing import MinMaxScaler

# Initialize SelectKBest
kbest = SelectKBest(score_func=chi2, k=10)

# Scale features
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Fit SelectKBest
kbest.fit(X_scaled, y)

# Get top features
best_features = np.argsort(kbest.scores_)[-2:]  # Selecting top 2 features
features_to_delete = np.argsort(kbest.scores_)[:-2]
X = np.delete(X, features_to_delete, axis=1)

print(f"Shape after feature selection: {X.shape}")
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
				
						from sklearn.feature_selection import SelectKBest, chi2
from sklearn.preprocessing import MinMaxScaler
 
# Initialize SelectKBest
kbest = SelectKBest(score_func=chi2, k=10)
 
# Scale features
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)
 
# Fit SelectKBest
kbest.fit(X_scaled, y)
 
# Get top features
best_features = np.argsort(kbest.scores_)[-2:]  # Selecting top 2 features
features_to_delete = np.argsort(kbest.scores_)[:-2]
X = np.delete(X, features_to_delete, axis=1)
 
print(f"Shape after feature selection: {X.shape}")
					
				
			
		



输出：




		
		
			
			
Java
			
			Shape after feature selection: (142193, 2)
			
				
					
				
					1
				
						Shape after feature selection: (142193, 2)
					
				
			
		



特征缩放

缩放确保所有特征对模型性能的贡献相同。

标准化

将数据转换为均值为零，标准差为一。





		
		
			
			
Java
			
			from sklearn.preprocessing import StandardScaler

sc = StandardScaler(with_mean=False)
sc.fit(X_train)

X_train = sc.transform(X_train)
X_test = sc.transform(X_test)
			
				
					
				
					1
2
3
4
5
6
7
				
						from sklearn.preprocessing import StandardScaler
 
sc = StandardScaler(with_mean=False)
sc.fit(X_train)
 
X_train = sc.transform(X_train)
X_test = sc.transform(X_test)
					
				
			
		



训练模型

我们将比较两种分类模型：K-最近邻（KNN）和逻辑回归。

训练-测试拆分

将数据拆分为训练集和测试集，确保我们能够有效地评估模型性能。





		
		
			
			
Java
			
			from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.20, random_state=1
)

print(f"Training set shape: {X_train.shape}")
print(f"Testing set shape: {X_test.shape}")
			
				
					
				
					1
2
3
4
5
6
7
8
				
						from sklearn.model_selection import train_test_split
 
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.20, random_state=1
)
 
print(f"Training set shape: {X_train.shape}")
print(f"Testing set shape: {X_test.shape}")
					
				
			
		



输出：




		
		
			
			
Java
			
			Training set shape: (113754, 2)
Testing set shape: (28439, 2)
			
				
					
				
					1
2
				
						Training set shape: (113754, 2)
Testing set shape: (28439, 2)
					
				
			
		



K-最近邻（KNN）

KNN是一种简单的基于实例的学习算法，用于分类和回归。





		
		
			
			
Java
			
			from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Initialize KNN with 3 neighbors
knnClassifier = KNeighborsClassifier(n_neighbors=3)
knnClassifier.fit(X_train, y_train)

# Predict and evaluate
y_pred_knn = knnClassifier.predict(X_test)
knn_accuracy = accuracy_score(y_pred_knn, y_test)
print(f"KNN Accuracy: {knn_accuracy:.2%}")
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
				
						from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score
 
# Initialize KNN with 3 neighbors
knnClassifier = KNeighborsClassifier(n_neighbors=3)
knnClassifier.fit(X_train, y_train)
 
# Predict and evaluate
y_pred_knn = knnClassifier.predict(X_test)
knn_accuracy = accuracy_score(y_pred_knn, y_test)
print(f"KNN Accuracy: {knn_accuracy:.2%}")
					
				
			
		



输出：




		
		
			
			
Java
			
			KNN Accuracy: 80.03%
			
				
					
				
					1
				
						KNN Accuracy: 80.03%
					
				
			
		



逻辑回归

逻辑回归是一种强大的二元分类算法，估计二元结果的概率。





		
		
			
			
Java
			
			from sklearn.linear_model import LogisticRegression

# Initialize Logistic Regression
LRM = LogisticRegression(random_state=0, max_iter=200)
LRM.fit(X_train, y_train)

# Predict and evaluate
y_pred_lr = LRM.predict(X_test)
lr_accuracy = accuracy_score(y_pred_lr, y_test)
print(f"Logistic Regression Accuracy: {lr_accuracy:.2%}")
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
				
						from sklearn.linear_model import LogisticRegression
 
# Initialize Logistic Regression
LRM = LogisticRegression(random_state=0, max_iter=200)
LRM.fit(X_train, y_train)
 
# Predict and evaluate
y_pred_lr = LRM.predict(X_test)
lr_accuracy = accuracy_score(y_pred_lr, y_test)
print(f"Logistic Regression Accuracy: {lr_accuracy:.2%}")
					
				
			
		



输出：




		
		
			
			
Java
			
			Logistic Regression Accuracy: 82.97%
			
				
					
				
					1
				
						Logistic Regression Accuracy: 82.97%
					
				
			
		



评估模型性能

KNN和逻辑回归在数据集上都提供了相当的准确性，但在这种情况下，逻辑回归的表现优于KNN。


    
        模型
        准确率
    
    
        K-最近邻
        80.03%
    
    
        逻辑回归
        82.97%
    


超参数调优

优化超参数可以进一步提高模型性能。对于逻辑回归，参数如C（正则化强度的倒数）和solver可以进行调优。同样，KNN的n_neighbors也可以变化。

示例：使用GridSearchCV进行逻辑回归





		
		
			
			
Java
			
			from sklearn.model_selection import GridSearchCV

# Define parameter grid
param_grid = {
    'C': [0.01, 0.1, 1, 10, 100],
    'solver': ['liblinear', 'lbfgs']
}

# Initialize GridSearchCV
grid = GridSearchCV(LogisticRegression(max_iter=200), param_grid, cv=5)
grid.fit(X_train, y_train)

print(f"Best Parameters: {grid.best_params_}")
print(f"Best Cross-Validation Accuracy: {grid.best_score_:.2%}")
			
				
					
				
					1
2
3
4
5
6
7
8
9
10
11
12
13
14
				
						from sklearn.model_selection import GridSearchCV
 
# Define parameter grid
param_grid = {
    'C': [0.01, 0.1, 1, 10, 100],
    'solver': ['liblinear', 'lbfgs']
}
 
# Initialize GridSearchCV
grid = GridSearchCV(LogisticRegression(max_iter=200), param_grid, cv=5)
grid.fit(X_train, y_train)
 
print(f"Best Parameters: {grid.best_params_}")
print(f"Best Cross-Validation Accuracy: {grid.best_score_:.2%}")
					
				
			
		



输出：




		
		
			
			
Java
			
			Best Parameters: {'C': 1, 'solver': 'lbfgs'}
Best Cross-Validation Accuracy: 83.25%
			
				
					
				
					1
2
				
						Best Parameters: {'C': 1, 'solver': 'lbfgs'}
Best Cross-Validation Accuracy: 83.25%
					
				
			
		



实现最佳参数：





		
		
			
			
Java
			
			# Initialize Logistic Regression with best parameters
best_lr = grid.best_estimator_
best_lr.fit(X_train, y_train)

# Predict and evaluate
y_pred_best_lr = best_lr.predict(X_test)
best_lr_accuracy = accuracy_score(y_pred_best_lr, y_test)
print(f"Optimized Logistic Regression Accuracy: {best_lr_accuracy:.2%}")
			
				
					
				
					1
2
3
4
5
6
7
8
				
						# Initialize Logistic Regression with best parameters
best_lr = grid.best_estimator_
best_lr.fit(X_train, y_train)
 
# Predict and evaluate
y_pred_best_lr = best_lr.predict(X_test)
best_lr_accuracy = accuracy_score(y_pred_best_lr, y_test)
print(f"Optimized Logistic Regression Accuracy: {best_lr_accuracy:.2%}")
					
				
			
		



输出：




		
		
			
			
Java
			
			Optimized Logistic Regression Accuracy: 83.00%
			
				
					
				
					1
				
						Optimized Logistic Regression Accuracy: 83.00%
					
				
			
		



结论

在本指南中，我们成功地在Python中实现了逻辑回归模型，展示了从数据预处理到模型评估的整个机器学习流程。通过处理缺失数据、编码分类变量、选择相关特征和缩放，我们优化了数据集以实现卓越的模型性能。此外，将逻辑回归与KNN进行比较，突显了各算法的优势，在这种情境下，逻辑回归表现稍优。

关键要点：

    数据预处理：对于实现高模型准确性至关重要。
    特征选择：有助于减少过拟合并提高性能。
    模型比较：始终比较多种模型以识别最佳表现者。
    超参数调优：优化模型性能的关键步骤。


掌握这些技术，构建适合您特定数据集和需求的强大且高效的分类模型。



关键词：逻辑回归, Python, Scikit-Learn, 机器学习, 数据预处理, 分类模型, K-最近邻, 特征选择, 超参数调优, 数据科学教程

Meta Description：学习如何使用Scikit-Learn在Python中实现逻辑回归。本全面指南涵盖数据预处理、处理缺失值、特征选择和模型评估，并比较逻辑回归与KNN以实现最佳性能。
Date	Location	MinTemp	MaxTemp	Evaporation	...	RainToday	RainTomorrow
2017-06-20	Uluru	3.5	21.8	NaN	...	No	No
2017-06-21	Uluru	2.8	23.4	NaN	...	No	No
2017-06-22	Uluru	3.6	25.3	NaN	...	No	No
2017-06-23	Uluru	5.4	26.9	NaN	...	No	No
2017-06-24	Uluru	7.8	27.0	NaN	...	No	No