Please enable JavaScript.
Coggle requires JavaScript to display documents.
Heroes of Deep Learning - Coggle Diagram
Heroes of Deep Learning
Geoffrey Hinton
-
Advices
- Read enough to develop intuitions
- Discover something that everybody is doing wrong
- Work on it and never stop programming
-
Pieter Abbel
Deep RL Future
-
-
-
-
Transfer Learning: Reusing Learning, avoid training from scratch
Advice
- Try things out. Don't just watch videos.
- You can learn faster if you have somebody who's more experienced
-
Andrew NG
Every three months, try to check you hyperparameters in long projects.
-
-
-
Yoshua Bengion
-
Science Thoughts
There are mental construction that humans made, important concepts that we still need disentangle
Our systems right now make the kind of mistakes that suggests that hey have a very superficial understanding of the world
-
-
Advices
-
-
Implement things by yourself, try to derivate things yourself from first principles.
When you read something, try to wonder why people are doing this?
-
Review ICLR and continuous path in Optimization, Algebra, Probability and Calculus.
-
Andrej Karpathy
Advice
Go all the way down to the basic concepts and program them from scratch using basic libraries like numpy.
Move to fancy libraries like TensorFlow or PyTorch once you already understand what is happening under the hood.
Personal Life
He have a similar introduction (first exposure) to Deep Learning as me. He start with a class that catch his attention, but for the master he realize that ML was something that he wanted to do.
-
-
Future Thoughts in AI
-
Decompose the intelligence by sub-tasks and them try to unify them into a brain is not a good approach for AGI
It is better for an AGI to have a kind of single NN that is a complete dynamical system. But the challenges are to define objectives in such a way that after optimizing it you can get an intelligent behavior
-
-
Works
-
RBM are a kind of Generative Model, which is trying to model coupling distributions in the data. They don't use Backpropagation
-
Other Learning algorithms are Markov Chain Monte Carlo and Variational Learning (Not Scalable as BackProg)
-
Future Trends
- Deep Reinforement Learning. How to scale and communicate with each others.
- Reasoning in NLP: Reasons. Read text and answer questions
- Being able to learn from few examples: One-shot learning and Transfer Learning (to solve tasks really quickly).
Advices
Try different things, not be afraid to try new things (innovate)
-
Academy vs Industry:
- Academy: more freedom to work on long term problems or crazy problems.
- You can impact millions of users and have much resources
