Please enable JavaScript.

Coggle requires JavaScript to display documents.

Instance Normalization(2016) - Coggle Diagram

- - - - Batch Normalization
        
        $$\mu_i= \frac{1}{HWT} \sum_{t=1}^T \sum_{l=1}^W \sum_{m=1}^H x_{tilm}$$
        
        $$\sigma^2_i= \frac{1}{HWT} \sum_{t=1}^T \sum_{l=1}^W \sum_{m=1}^H (x_{tilm} -\mu_i)^2$$
        
        $$y_{tijk}=\frac{x_{tilk}-\mu_i} {\sqrt{\sigma_i^2+\epsilon}}$$
      - Layer Normalization
        
        $$y_{tijk}=\frac{x_{tilk}-\mu_{ti}} {\sqrt{\sigma_{ti}^2+\epsilon}}$$
        
        $$\mu_{ti}= \frac{1}{HW} \sum_{l=1}^W \sum_{m=1}^H x_{tilm}$$
        
        $$\sigma^2_{ti}= \frac{1}{HW} \sum_{l=1}^W \sum_{m=1}^H (x_{tilm} -\mu_{ti})^2$$
  - - - $$y_{tijk}=\frac{x_{tijk}} {\sum_{l=1}^W \sum_{m=1}^Hx_{tilm}}$$