前言

用Sklearn常用的Ensemble算法對當當熱銷書評論進行分類實踐。

關于集成算法概念可以看這篇文章 總結Bootstraping、Bagging和Boosting

先看一下這篇文章樸素貝葉斯分類算法實踐，本文主要還是用當當評論數據做的分析。關于代碼部分一些細節在樸素貝葉斯分類算法實踐已經詳細的解釋了。

• 完整代碼查看：https://github.com/xhades/rates_classify/tree/master/rates_classify
• 訓練數據下載地址：https://pan.baidu.com/s/1kVOS39l

正文

RandomForest

sklearn RandomForestClassifier文檔地址

代碼

import numpy as np
from numpy import array, argmax, reshape
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
import pickle
from sklearn.ensemble import RandomForestClassifier as RDF
np.set_printoptions(threshold=np.inf)


# 訓練集測試集 3/7分割
def train(xFile, yFile):
    with open(xFile, "rb") as file_r:
        X = pickle.load(file_r)

    X = reshape(X, (212841, -1))  # reshape一下 （212841, 30*128）
    # 讀取label數據，并且encodig
    with open(yFile, "r") as yFile_r:
        labelLines = [_.strip("\n") for _ in yFile_r.readlines()]
    values = array(labelLines)
    labelEncoder = LabelEncoder()
    integerEncoded = labelEncoder.fit_transform(values)
    integerEncoded = integerEncoded.reshape(len(integerEncoded), 1)
    # print(integerEncoded)

    # 獲得label  編碼
    Y = integerEncoded.reshape(212841, )
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3, random_state=42)

    # 隨機森林分類器
    clf = RDF(criterion="gini")
    # criterion 可以使用"gini"或者"entropy"，前者代表基尼系數，后者代表信息增益。一般說使用默認的基尼系數"gini"就可以了，即CART算法。除非你更喜歡類似ID3, C4.5的最優特征選擇方法。

    clf.fit(X_train, Y_train)

    # 測試數據
    predict = clf.predict(X_test)
    count = 0
    for p, t in zip(predict, Y_test):
        if p == t:
            count += 1
    print("RandomForest Accuracy is:", count/len(Y_test))


if __name__ == "__main__":
    xFile = "Res/char_embedded.pkl"
    yFile = "data/label.txt"
    print("Start Training.....")
    train(xFile, yFile)
    print("End.....")

主要的參數說明

• criterion 可以使用"gini"或者"entropy"，前者代表基尼系數，后者代表信息增益。一般說使用默認的基尼系數"gini"就可以了，即CART算法。除非你更喜歡類似ID3, C4.5的最優特征選擇方法。
• 其他參數都用默認，以后再更新 =。=

結果

Start Training.....
RandomForest Accuracy is: 0.9258453009255634
End.....

最終結果大概92.6%左右的準確率

梯度提升算法GradientBoostingClassifier

sklearn GradientBoostingClassifier 文檔地址

Boosting不斷串行地迭代弱學習器最終形成一個強學習器，這點和Bagging并行的 方式不同，所以在用梯度提升算法時耗時非常長

代碼

import numpy as np
from numpy import array, argmax, reshape
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
import pickle
from sklearn.ensemble import GradientBoostingClassifier as GBC

np.set_printoptions(threshold=np.inf)


# 訓練集測試集 3/7分割
def train(xFile, yFile):
    with open(xFile, "rb") as file_r:
        X = pickle.load(file_r)

    X = reshape(X, (212841, -1))  # reshape一下 （212841, 30*128）
    # 讀取label數據，并且Encoding
    with open(yFile, "r") as yFile_r:
        labelLines = [_.strip("\n") for _ in yFile_r.readlines()]
    values = array(labelLines)
    labelEncoder = LabelEncoder()
    integerEncoded = labelEncoder.fit_transform(values)
    integerEncoded = integerEncoded.reshape(len(integerEncoded), 1)
    # print(integerEncoded)

    # 獲得label 編碼
    Y = integerEncoded.reshape(212841, )
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3, random_state=42)

    # 梯度提升分類器
    clf = GBC(loss="deviance", subsample=0.8, criterion="friedman_mse")

    clf.fit(X_train, Y_train)

    # 測試數據
    predict = clf.predict(X_test)
    count = 0
    for p, t in zip(predict, Y_test):
        if p == t:
            count += 1
    print("GradientBoosting  Accuracy is:", count/len(Y_test))


if __name__ == "__main__":
    xFile = "Res/char_embedded.pkl"
    yFile = "data/label.txt"
    print("Start Training.....")
    train(xFile, yFile)
    print("End.....")

主要的參數說明

• subsample 數據隨機抽樣對決策樹進行訓練，這個參數設置比1小即可，具體數值需要在“調參”過程中發現最優
• 其他參數日后再(tai)整(lan)理(le)
• 源碼中的默認參數設置

    _SUPPORTED_LOSS = ('deviance', 'exponential')

    def __init__(self, loss='deviance', learning_rate=0.1, n_estimators=100,
                 subsample=1.0, criterion='friedman_mse', min_samples_split=2,
                 min_samples_leaf=1, min_weight_fraction_leaf=0.,
                 max_depth=3, min_impurity_split=1e-7, init=None,
                 random_state=None, max_features=None, verbose=0,
                 max_leaf_nodes=None, warm_start=False,
                 presort='auto'):

        super(GradientBoostingClassifier, self).__init__(
            loss=loss, learning_rate=learning_rate, n_estimators=n_estimators,
            criterion=criterion, min_samples_split=min_samples_split,
            min_samples_leaf=min_samples_leaf,
            min_weight_fraction_leaf=min_weight_fraction_leaf,
            max_depth=max_depth, init=init, subsample=subsample,
            max_features=max_features,
            random_state=random_state, verbose=verbose,
            max_leaf_nodes=max_leaf_nodes,
            min_impurity_split=min_impurity_split,
            warm_start=warm_start,
            presort=presort)

結果

Start Training.....
GradientBoosting  Accuracy is: 0.8833727467777551
End.....

最終準確率88.3%左右