Please enable JavaScript.
Coggle requires JavaScript to display documents.
BN (BN 優點, 一些問題（Introduction）, Internal Covariant Shift, BN 缺點,…
BN
BN 優點
can be trained with saturating nonlinearities
allows us to use much higher learning rates, faster convergence
acts as a regularizer（Dropout）
less careful about initialization
Makes deep networks much easier to train
一些問題（Introduction）
Careful parameter initialization
hard to train model in saturating nonlinearities
learning rates 不能太大
不 normalize 的話，hard to train，descent 方向會亂跳
Internal Covariant Shift
到越 deep 的地方，分佈會越來越不 standard normal distribution
BN 缺點
Behaves differently during training and testing
實驗結果（Experiments）
那兩張圖
BN 實作細節
講一下 hard to represent 那個
為什麼要 normalize
李宏毅 error surface 那張圖 +
https://pse.is/3jys3s