Please enable JavaScript.

Coggle requires JavaScript to display documents.

Backpropagation ((Om \(C=\frac{1}{2}\sum_{j}\left(t_{j}-a…

- - - - \(\boldsymbol{\delta}^{L}=\color{orange}{-}\color{red}{\sigma'(\boldsymbol{z}^{L})}\color{orange}{(\boldsymbol{t}-\boldsymbol{a}^{L})}\)
    - - \(\color{orange}{\textrm{Ju mer fel outputen är desto mer ändras det}}\)
        
        \(\color{red}{\textrm{Ju mer saturerad aktiveringsfunktionen är desto långsammare ändras vi}}\)
        
        \(\textrm{Ju mer aktiverad neuronen är desto mer ändras det}\)
    - - \(\boldsymbol{\delta}^{L-1}=\color{orange}{-}\color{red}{\sigma'(\boldsymbol{z}^{L-1})}\odot\left(W^{L}\right)^{T}(\color{red}{\sigma'(\boldsymbol{z}^{L})}\color{orange}{(\boldsymbol{t}-\boldsymbol{a}^{L})})\)
        \(\delta_j^{L-1}=\color{orange}{-}\color{red}{\sigma'(z_j^{L-1})}\sum_{k}(\color{red}{\sigma'(z_{k}^{L})}\color{orange}{(t_{k}-a_{k}^{L})}w_{kj}^{L})\)
      - \(\frac{\partial C}{\partial w_{jk}^{L-1}}=\color{orange}{-(t_{j}-a_{j}^{L})}\color{red}{\sigma'(z_{j}^{L})}a_{k}^{L-1}\)
      - Om bara en outputneuron, som på bilden:
        
        \(\delta_j^{L-1}=\color{orange}{-}\color{red}{\sigma'(z_j^{L-1})}\color{red}{\sigma'(z_{1}^{L})}\color{orange}{(t_{1}-a_{1}^{L})}w_{1j}^{L}\)
      - \(\delta_i^{L-2}=\color{orange}{-}\color{red}{\sigma'(z_i^{L-2})}\sum_{m}(\color{red}{\sigma'(z_m^{L-1})}\sum_{k}(\color{red}{\sigma'(z_{k}^{L})}\color{orange}{(t_{k}-a_{k}^{L})}w_{km}^{L})w_{mi}^{L-1})\)
        
        \(\sum_{k,m}\color{red}{\sigma'(z_{j}^{L-2})\sigma'(z_{k}^{L-1})\sigma'(z_{m}^{L})}w_{kj}^{L-1}w_{mk}^{L}\color{orange}{(-(t_{m}-a_{m}^{L}))}\)
        Låt oss använda index \(n_{L-2},n_{L-1}\) och \(n_{L}\) istället för i,j,k (\(n\) för neuron)
        Vi får då
        \(\sum_{n_{L-2},...,n_{L}}\color{red}{\sigma'(z_{n_{L-2}}^{L-2})\sigma'(z_{n_{L-1}}^{L-1})\sigma'(z_{n_{L}}^{L})}w_{n_{L-1}n_{L-2}}^{L-1}w_{n_{L}n_{L-1}}^{L}\color{orange}{(-(t_{n_{L}}-a_{n_{L}}^{L}))}\)
        
        och generellt
        \(\delta_{n_{l}}^{l}=\sum_{n_{l+1},...,n_{L}}\color{red}{\sigma'(z_{n_{l}}^{l})\sigma'(z_{n_{l+1}}^{l+1})\cdots\sigma'(z_{n_{L}}^{L})}w_{n_{l+1}n_{l}}^{l+1}\cdots w_{n_{L}n_{L-1}}^{L}\color{orange}{(-(t_{n_{L}}-a_{n_{L}}^{L}))}\)