Please enable JavaScript.

Coggle requires JavaScript to display documents.

Deep CNN (AlexNet (Five major contributions (Reducing overfitting (Data…

- - - - Rectified Linear Units(ReLU)
        
        To replace tanh or sigmoid
        
        Faster convergence and lower gradient vanishing
    - - 池化過程中相鄰兩點會重疊
    - - Data augmentation
        
        人工收集資料較佳
        
        成本高
      - dropout
- - - - Vgg16
      - Vgg19
  - - - 所使用的參數較少
      - 非線性的成分較高
  - - - Spatial contest is important
- - - - 不同數量層的inception間
        
        加入maxpooling
        
        降低解析度
      - Inception前
        
        使用traditional convollutions
    - - 使用global average pooling
        
        考慮到多張Feature map的平均
  - - - choose mode filrers
        
        3x3
        
        5x5
        
        取較大範圍的資訊
        
        1x1
        
        取較細微範圍的資訊
      - Max pooling
        
        3x3
      - 透過linear decrease 降低參數量
        
        使用1x1 的filters
        
        時間成本的考量
    - - intermediate layers
      - intermediate losses
- - - - gradient exploding / vanishing
        
        越深不見得越好!!
        
        解決方法 ResNet!!
        
        stacking residual modules
        
        batch normalization
        
        Global max pooling
  - - - 學習residual
        
        short cut connection
        
        還原解析度
    - - Accelerate the training of deep networks
      - Reduce the risk of vanishing gradient
      - Less parameters in a single module
      - Increase depth of the network
      - Achieve higher accuracy in many vision applications
- - - - Encourage feature reuse
      - Reduce the number of parameters
      - alleviate vanishing gradient
      - Strengthen feature propagation