{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  8.3 不同卷积网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.3.1 CNN 结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经典的卷积网络结构，是组合卷积层、池化层和全连接层。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-9.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最近几年中，不断出现各种新颖的网络结构，如 GoogLeNet、ResNet。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.3.2 GoogLeNet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GoogLeNet的基本组成单元是 Inception module。不同于单一的卷积层，Inception module 会分成多路不同分辨率卷积和池化的结合："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-10.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GoogLeNet的前两层是大的卷积核和步长，降低了十六倍的计算量，然后进行局部的归一化。同样的两个卷积层之后，是9层Inception module的堆叠。然后是全局平均池化，输出到softmax的一千个类别。其中更琐碎的底层损失函数没有在图中展示。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-11.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.3.3 ResNet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差网络（Residual Network）赢得了 ILSVRC 2015 的冠军，使用了超越以往的152层的网络。该网络使用了隔层连接（skip connections）的技术，不同于直接计算目标函数，而是使用计算残差的方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-12.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原本是要计算的 $h(x)$ ，想在变为计算 $h(x)-x$ 。短接连接的引入，使得网络可以不再严重依赖前一层的激活值就可以正常计算梯度。参数值与激活值互为梯度："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-13.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差网络就是由这样一系列的短接单元组成了极深的深度网络，甚至可以达到1000层："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/13-14.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "随着深度的不断加深，特征图的数量也不断增多，其中还包含了批量归一化层。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}