Please enable JavaScript.
Coggle requires JavaScript to display documents.
COMPGI22 - Advanced DL & RL (DL (Attention & memory in DL…
COMPGI22 - Advanced DL & RL
DL
Attention & memory in DL
implicit attention
NN Jacobian - set of derivatives
explicit attention to external data
hard attention
attention model - operates on some extra data to create a fixed-size 'glimpse'
train attention model w RL
soft attention
train end-to-end w backprop
location attention
associative attention by content
introspective attention - attention to NN's internal state/memory
neural turing machine (2014)
addressing by content
addressing by location
NTM copy algorithm
copy N times
N-gram inference
priority sort
differentiable neural computers (2016) - NTM w new attention mechanisms for memory access
new memory allocation
searching by time - temporal order
graph experiment - london underground map
differentiable visual attention (DRAW, 2015)
spatial transformer networks (2015)
types of neural attention models - summary
content
location
visual
temporal