Please enable JavaScript.

Coggle requires JavaScript to display documents.

Deep Learning (:apple:神经网络 (:one:NN (:banana:激活函数 (:one:sigmoid (:fire…

- - - - :one:易陷入局部最优解
        
        :maple_leaf:SGD, Adam, Resterov等优化器理论上可以解决
      - :two:易产生梯度弥散/爆炸
      - :three:参数指数级增加
    - - :one:sigmoid
        
        :fire:值域是[0,1]
        
        :pencil2:\( sigmoid(x) = \dfrac{e^x}{e^x + 1} \)
        
        \( \dfrac{\partial sigmoid(x)}{\partial x} = sigmoid(x) (1 - sigmoid(x)) \)
      - :two:tanh
        
        :fire:值域是[-1,1]
        
        :pencil2:\( \tanh(x) = \dfrac{e^{x} - e^{-x}}{e^{x} + e^{-x}} \)
        
        \( \dfrac{\partial {tanhx}}{\partial x} = 1 - tanh^2{x} \)
      - :three:softmax
      - :four:relu
      - :question:为什么使用这些函数作为激活函数
        
        :explode:比如sigmoid和tanh,他们的导数都可以用自身来表示,有利于编程的梯度实现
    - - :maple_leaf:我觉得应该是每个神经元1个
- - - - :explode:1*1 + 3*3 + 1*1的网络,并且前一层的输入可以直接反馈到输出,而不需要经过卷积的操作
- - - - :arrow_right:前面的filters数量小,因为可能只需要学习线,点这样简单的概念
  - - - :palm_tree:如果某个分类的数据量少,那么这个分类相对于其他分类其实是弱的
        
        :icecream:头发和性别的简单分类
        
        :one:1 hidden layer
        
        :two:multiple hidden layers