{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 5.2 分类函数" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5.2.1 DecisionTreeClassifier(决策树)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Decision Trees (DTs)创建一种模型从数据特征中学习简单的决策规则来预测一个目标变量的值。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "class sklearn.tree.DecisionTreeClassifier(criterion='gini', splitter='best', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, class_weight=None, presort=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "参数解释(Parameters):\n", "\n", " criterion : string, optional (default=”gini”). # 度量节点分裂质量的函数. “gini”表示用基尼指数的度量方式， \n", " “entropy”表示用信息增益的度量方式。\n", " \n", " splitter : string, optional (default=”best”) # 用于选择每个节点上分割的策略. “best”表示the best split，\n", " “random” 表示the best random split.\n", " \n", " max_depth : int or None, optional (default=None). # 树的最大深度. 如果设置为None, 那么树的节点将会扩展，直至所有叶子节点是纯净的，\n", " 或者所有叶子节点包含的样本数少于 min_samples_split samples.\n", " \n", " min_samples_split : int, float, optional (default=2) # 节点进行划分所需要的最少样本数：\n", " If int, then consider min_samples_split as the minimum number.\n", " If float, then min_samples_split is a percentage and ceil(min_samples_split * n_samples) \n", " are the minimum number of samples for each split.\n", " \n", " min_samples_leaf : int, float, optional (default=1) # 在叶子节点所需的最小样本数:\n", " If int, then consider min_samples_leaf as the minimum number.\n", " If float, then min_samples_leaf is a percentage and ceil(min_samples_leaf * n_samples) are \n", " the minimum number of samples for each node.\n", " \n", " min_weight_fraction_leaf : float, optional (default=0.) # 需要在叶节点上的权重（所有输入样本）的总和的最小加权百分率。\n", " 当不提供样本权重时，样本具有相同的权重。\n", " \n", " max_features : int, float, string or None, optional (default=None). # 当寻找最优分裂时，考虑的特征数量：\n", " If int, then consider max_features features at each split.\n", " If float, then max_features is a percentage and int(max_features * n_features) features are considered at each split.\n", " If “auto”, then max_features=sqrt(n_features).\n", " If “sqrt”, then max_features=sqrt(n_features).\n", " If “log2”, then max_features=log2(n_features).\n", " If None, then max_features=n_features.\n", " \n", " random_state : int, RandomState instance or None, optional (default=None). # \n", " If int, random_state 作为随机数生成器的种子。\n", " If RandomState instance, random_state is 随机数生成器; \n", " If None, the random number generator is the RandomState instance used by np.random.\n", " \n", " max_leaf_nodes : int or None, optional (default=None). # Grow a tree with max_leaf_nodes in best-first fashion. \n", " Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.\n", " \n", " min_impurity_decrease : float, optional (default=0.). # 如果这次分裂导致节点不纯度的下降大于或等min_impurity_decrease，该节点将会分裂。\n", " \n", " min_impurity_split : float, # 树生长过程中，提前停止的阈值。 如果一个节点的不纯度高于该阈值，节点将会分裂；否则，该节点为叶节点。\n", " \n", " class_weight : dict, list of dicts, “balanced” or None, default=None. # Weights associated with classes in the form \n", " {class_label: weight}. If not given, all classes are supposed to have weight one. For multi-output problems, a list of dicts can \n", " be provided in the same order as the columns of y.\n", " \n", " presort : bool, optional (default=False). # 是否要对数据进行预分类，以加快训练中找到最佳分割。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "属性(Attributes)：\n", "\n", " classes_ : array of shape = [n_classes] or a list of such arrays. # 类别标签(single output problem), \n", " or 类别标签的数组List(multi-output problem).\n", " feature_importances_ : array of shape = [n_features]. # 特征重要性，值越大表示特征越重要，体现了以该特征进行节点分裂，\n", " 不纯度下降越快。\n", " max_features_ : int, # 推测出的最大特征数量.\n", " n_classes_ : int or list # The number of classes (for single output problems), \n", " or a list containing the number of classes for each output (for multi-output problems).\n", " n_features_ : int # 当进行拟合时，所用的特征数量.\n", " n_outputs_ : int # The number of outputs when fit is performed.\n", " tree_ : Tree object # 训练拟合之后的树对象." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "方法(Methods):\n", "\n", " apply(X[, check_input])\t # 返回每个样本被预测为的叶子的索引。\n", " decision_path(X[, check_input])\t # 返回树的决策路径。\n", " fit(X, y[, sample_weight, check_input, …])\t # Build a decision tree classifier from the training set (X, y).\n", " get_params([deep])\t # 获得this estimator的参数。\n", " predict(X[, check_input])\t # Predict class or regression value for X.\n", " predict_log_proba(X)\t # Predict class log-probabilities of the input samples X.\n", " predict_proba(X[, check_input])\t # Predict class probabilities of the input samples X.\n", " score(X, y[, sample_weight])\t # Returns the mean accuracy on the given test data and labels.\n", " set_params(**params)\t # Set the parameters of this estimator." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5.2.2 AdaBoostClassifier(串行集成-AdaBoost)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "AdaBoost 的核心思想是用反复修正数据的权重来训练一系列的弱学习器(一个弱学习器模型仅仅比随机猜测好一点, 比如一个简单的决策树),由这些弱学习器的预测结果通过加权投票(或加权求和)的方式组合, 得到我们最终的预测结果。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "class sklearn.ensemble.AdaBoostClassifier(base_estimator=None, n_estimators=50, learning_rate=1.0, algorithm='SAMME.R', random_state=None)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "参数解释(Parameters):\n", "\n", " base_estimator : object, optional (default=DecisionTreeClassifier). # 设置基分类器。\n", " \n", " n_estimators : integer, optional (default=50). # 在boosting 终止时，基分类器的最大数量。 \n", " 在极好的拟合情况下，学习过程被提前停止。\n", " \n", " learning_rate : float, optional (default=1.). # 通过设置学利率，学习率缩减每个分类器的贡献. \n", " There is a trade-off between learning_rate and n_estimators.\n", " \n", " algorithm : {‘SAMME’, ‘SAMME.R’}, optional (default=’SAMME.R’). # If ‘SAMME.R’ then use the SAMME.R real boosting \n", " algorithm. base_estimator must support calculation of class probabilities. If ‘SAMME’ then use the SAMME discrete boosting\n", " algorithm. The SAMME.R algorithm typically converges faster than SAMME, achieving a lower test error with fewer \n", " boosting iterations.\n", " \n", " random_state : int, RandomState instance or None, optional (default=None). \n", " If int, random_state is the seed used by the random number generator; \n", " If RandomState instance, random_state is the random number generator; \n", " If None, the random number generator is the RandomState instance used by np.random\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "属性(Attributes)：\n", "\n", " estimators_ : list of classifiers # 拟合后的基分类器的集合。\n", " classes_ : array of shape = [n_classes] # 类别标签.\n", " n_classes_ : int # The number of classes.\n", " estimator_weights_ : array of floats # 在boosted 集成中，每个基分类器的权重。\n", " estimator_errors_ : array of floats # 在boosted集成中，每个基分类器的分类误差。\n", " feature_importances_ : array of shape = [n_features] # The feature importances if supported by the base_estimator" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "方法(Methods):\n", "\n", " decision_function(X)\t # 计算 X的决策函数值.\n", " fit(X, y[, sample_weight])\t # Build a boosted classifier from the training set (X, y).\n", " get_params([deep])\t # Get parameters for this estimator.\n", " predict(X)\t # Predict classes for X.\n", " predict_log_proba(X)\t # Predict class log-probabilities for X.\n", " predict_proba(X)\t # Predict class probabilities for X.\n", " score(X, y[, sample_weight])\t # Returns the mean accuracy on the given test data and labels.\n", " set_params(**params)\t # Set the parameters of this estimator.\n", " staged_decision_function(X)\t # Compute decision function of X for each boosting iteration.\n", " staged_predict(X)\t # Return staged predictions for X.\n", " staged_predict_proba(X)\t # Predict class probabilities for X.\n", " staged_score(X, y[, sample_weight])\t # Return staged scores for X, y." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5.2.3 RandomForestClassifier(并行集成-随机森林)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "随机森林就是通过集成学习的思想将多棵树集成的一种算法(并行集成)，它的基本单元是决策树。每棵决策树都是一个分类器，那么对于一个输入样本，N棵树会有N个分类结果。而随机森林集成了所有的分类投票结果，将投票次数最多的类别指定为最终的输出，这就是一种最简单的 Bagging 思想。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "class sklearn.ensemble.RandomForestClassifier(n_estimators=10, criterion='gini', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=0, warm_start=False, class_weight=None)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "参数解释(Parameters):\n", "\n", " n_estimators : integer, optional (default=10). # 森林中的树的棵数。\n", " criterion : string, optional (default=”gini”) # 节点分裂准则。\n", " max_features : int, float, string or None, optional (default=”auto”) # 解释同决策树\n", " max_depth : integer or None, optional (default=None)\n", " min_samples_split : int, float, optional (default=2)\n", " min_samples_leaf : int, float, optional (default=1)\n", " min_weight_fraction_leaf : float, optional (default=0.)\n", " max_leaf_nodes : int or None, optional (default=None)\n", " min_impurity_split : float,\n", " min_impurity_decrease : float, optional (default=0.)\n", " bootstrap : boolean, optional (default=True) # 在构建树时是否使用引导样本(bootstrap samples)。\n", " oob_score : bool (default=False) # 是否使用out-of-bag samples来估计泛化精度。\n", " n_jobs : integer, optional (default=1) # 同时运行和预测的并行作业数量。如果设为-1，那么作业的数量被设置为内核数量。\n", " random_state : int, RandomState instance or None, optional (default=None)\n", " verbose : int, optional (default=0) # 控制树构建过程的冗长。\n", " warm_start : bool, optional (default=False) # 当设置为true时，重用前一个调用的解决方案来拟合，并将更多的基分类器添加到集合中，\n", " 否则，只需拟合一个完整的新的林。\n", " class_weight : dict, list of dicts, “balanced”" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "属性(Attributes)：\n", "\n", " estimators_ : list of DecisionTreeClassifier # 拟合完成后，随机森林中树对象的集合。\n", " classes_ : array of shape = [n_classes] or a list of such arrays\n", " n_classes_ : int or list\n", " n_features_ : int\n", " n_outputs_ : int # The number of outputs when fit is performed.\n", " feature_importances_ : array of shape = [n_features]\n", " oob_score_ : float\n", " oob_decision_function_ : array of shape = [n_samples, n_classes] # 在训练集上用袋外估计计算决策函数。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "方法(Methods):\n", "\n", " apply(X)\t # 对于X，运用森林中的一棵树, 返回叶节点的索引。\n", " decision_path(X)\t # Return the decision path in the forest\n", " fit(X, y[, sample_weight])\t # Build a forest of trees from the training set (X, y).\n", " get_params([deep])\t # Get parameters for this estimator.\n", " predict(X)\t # Predict class for X.\n", " predict_log_proba(X)\t # Predict class log-probabilities for X.\n", " predict_proba(X)\t # Predict class probabilities for X.\n", " score(X, y[, sample_weight])\t # Returns the mean accuracy on the given test data and labels.\n", " set_params(**params)\t # Set the parameters of this estimator." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }