{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.1 算法原理\n",
    "\n",
    "## 3.1.1 点到分类超平面距离\n",
    "\n",
    "### 3.1.1.1 距离公式\n",
    "![1](picture/3111.png)\n",
    "\n",
    "### 3.1.1.2 证明过程\n",
    "![2](picture/3112.png)\n",
    "\n",
    "## 3.1.2 损失函数\n",
    "![3](picture/312.png)\n",
    "\n",
    "## 3.1.3 梯度下降\n",
    "![4](picture/313.png)\n",
    "\n",
    "### 3.1.3.1 代数描述\n",
    "![5](picture/3131.png)\n",
    "\n",
    "### 3.1.3.2 问题示例\n",
    "![6](picture/3132.png)\n",
    "\n",
    "### 3.1.3.3 算法调优\n",
    "![7](picture/3133.png)\n",
    "\n",
    "## 3.1.4 正则化\n",
    "![8](picture/314.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1.5 实验要求\n",
    "\n",
    "（1）根据不同损失函数的定义绘制训练样本的（归一化）损失值。\n",
    "\n",
    "（2）根据不同损失函数的定义绘制假设测试样本分别为正类或负类的（归一化）损失值。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1.6 损失函数\n",
    "\n",
    "    损失函数与点到分类边界的距离有关：df=clf.decision_function(X)\n",
    "    hinge    :f=np.where(df < 1, 1 - df, 0)        \n",
    "    perceptron:f=-np.minimum(df, 0) <br>\n",
    "    log     :f=np.log2(1 + np.exp(-df)) \n",
    "    squared_h :f=np.where(df< 1 ,1-df,0)^2  \n",
    "    modified_h:f=modified_huber_loss(df, 1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1.7 各损失函数的特点\n",
    "\n",
    "    Hinge loss ： margin 内有损失 边界的支持向量决定边界\n",
    "    Perceptron ： 分错有损失\n",
    "    Log loss_  ： 整体样本有损失  所有样本共同决定分类边界\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1.8 损失函数的作用\n",
    "\n",
    "损失函数估量模型的预测值与真实值的不一致程度---预测错误的程度。<br>\n",
    "损失函数越小，模型的鲁棒性就越好。<br>\n",
    "损失函数是经验风险函数的核心，也是结构风险函数重要组成部分，包括了经验风险项和正则项。<br>\n",
    "    \n",
    "损失函数度量模型一次预测的好坏，风险函数（期望损失）度量平均意义下模型的好坏。<br>\n",
    "参数越多，模型越复杂，而越复杂的模型越容易过拟合。过拟合就是说模型在训练数据上的效果远远好于在测试集上的性能。此时可以考虑正则化，通过设置正则项前面的hyper parameter，来权衡损失函数和正则项，减小参数规模，达到模型简化的目的，从而使模型具有更好的泛化能力。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "137px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}