Please enable JavaScript.
Coggle requires JavaScript to display documents.
Deep learning research timeline
(Inspired by this article) (1980s (1990s,…
Deep learning research timeline
(Inspired by this article)
1980s
-
Multilayer feedforward networks are universal approximators, 1989
Boltzmann machine, 1985
Representation learning graph + generative
Unsupervised learning
-
Clustering: Self organizing maps, Adaptive inference of latent factors
-
Conv + Pooling + NN with Back Propagation (Hand written zip code recognition, Yann LeCun, 1989
-
1990s
-
LSTM, 1997
Solves vanishing gradient problem using CEC and hence can be deep leveraging long term dependencies
(Less popular alternatives echo state network (ESN) and Hessian-free methods)
-
2000s
-
2010s
Imagenet winner using CNN 15.3% error, second at 26.2%, 2012
-
-
-
-
0.95% to 0.35% error rate on MNIST just by using simple Deep network trained on GPUs, 2010
-
-
-
-
-
-
-
-
RNN
For dealing with sense of time and memory in speech problems. (Backpropagation through time. But, doesn't handle long term dependencies well and learning is not optimal)
-
SVMs and Random Forests were born during the second AI winter?