Please enable JavaScript.

Coggle requires JavaScript to display documents.

activation function (:two:softmax (:fire:参数冗余性质 (:arrow_right:可以设置\(\theta…

- - - - :key:每次经过一个sigmoid,都会把定义域的变化压缩到值域的变化(无穷大->(0,1)) => forward后越来越小
- - - - :bread:ReLU在负半区的输出为0, 这些神经元不会产生梯度训练,所以就减少了参数的依赖,实现稀疏激活
  - - - :explode:先linear regression,再对一组取max
      - :fire:一组如果有多个elms的话,那么可以形成更加复杂的函数
        
        :arrow_right:近似模拟所有函数
        
        :icecream:模拟ReLU
        
        :icecream:更多elms
      - :star:本质是case-wise linear function
      - :bread:explanation
- - - - :link:Softmax回归
      - :star:softmax具有参数冗余性质
  - - - :warning:往往保留,使算法更清晰
  - - - :star:如果要求loss对\( w_h \)的偏导,softmax下输出的所有\( y_i \),也就是\( \dfrac{e^{z_i}}{\sum_k{e^{z_k}}} \)都要对\(z_h\)求偏导 => 是否等于h的都会加和起来
      - :link:Classification and Loss Evaluation - Softmax and Cross Entropy Loss