Please enable JavaScript.

Coggle requires JavaScript to display documents.

ML+Biology (Neurobiology/Plasticity (Papers (Solving the distal Reward…

- - - - Hebbian Learning?
  - - - Weight represents voltage estimate of the target neuron
      - Weights self-normalize
    - - a particular order of firing induces a synaptic change (+ or -), which enhanced if extracellular DA is present during the critical window of a few seconds.
      - \( \dot{c} = -c/\tau_c + \text{STDP}(\tau)\delta(t - t_{\text{pre}/\text{post}}) \)
      - \( \dot{s} = cd \)
      - \( \dot{d} = -d/\tau_d + \text{DA}(t) \)
    - - LIF neuron
      - Lateral inhibition -- max neuron takes over for t_inhibit, the rest are kept at 0
      - A target activity is defined for the neurons (i.e., a number of times an output neuron should spike over an extended period of time
        
        \( \frac{\delta X_{\text{th}}}{\delta t} = \gamma (A - T) \)
        
        A -- actual firing rate
        
        T -- target firing rate
        
        95% accuracy with 500 units on MNIST
    - - Inhibition
      - Homoeostasis
        
        \( v_{th} + \Theta \)
      - Learning
        
        \( \delta w = \alpha (x_{pre} - x_{tar})(w_{max} - w)^\mu \)
- - - - \( Q_{k+1} = Q_k + 1/k [R_k - Q_k] \)
    - - \( V(S_t) = V(S_t) + \alpha * [G_t - V(S_t)] \)
    - - \( V(S_t) = V(S_t) + \alpha [R_{t+1} + \gamma*V(S_{t+1}) - V(S_t)] \)
  - - - \( \delta w_{ij} = s_i s_j \)
    - - \( \delta E_k = \sum w_{ki}s_i - \Theta_k \)
      - Threshold
        
        unit is always "on"
        
        The link between unit i and threshold unit has \( -\Theta_i \) weight
      - \( \frac{\partial P_{a}}{\partial w_{ij}} = \frac{1}{T} [s_i^{a}s_j^{a} - p_{ij}] \)
      - \( \frac{\partial G}{\partial w_{ij}} = \frac{1}{T} [p_{ij}^a - p_{ij}] \)
        
        \( \delta w_{ij} = \epsilon * (p_{ij}^a - p_{ij}) \)
        
        running avg of prob?