Please enable JavaScript.
Coggle requires JavaScript to display documents.
Linear Regression (coggle for ML) (Optimization (Gradient, AdaGrad (sqrt…
Linear Regression (
coggle for ML
)
kernel method
solve non-linear
generic kernel (RBF)
Autometic kernel (Neural Network)
non convex (compared to SVM) gradient based learning
check Optimization methods
Convolution Neural network (Image)
AlexNet (ReLU, Dropout, LRN, GPU)
VGGNet (small conv size, deep vs efficiency ? )
Inception v1 (remove full connection layer, Inception layer)
ResNet (HighWay network, memorize some inputs)
Inception v2 (2 3x3 to replace 1 5x5, BN layer)
Inception v3 (three kinds inceptions)
Inception v4 ( + ResNet)
LSTM (gate)
How to improve
Depth
Width
new layer
architecture
NLP
LSTM
Probabilistic Language Models
Word2Vec
CBOW
Skip-Gram
Latent Semantic Analysis
Reinforcement Learning Environment State, Action, Reward
methods
Policy Network
Q-Learning (DQN by Google)
Applications
DeepMind
Self-Driven car
AlphaGo
Machine Control
Game Play
Benchmark
OpenAI (gym)
manually engineer kernel (decades ago)
Linear problem
Overfitting/ Underfitting
Regularization
Validation sets
bias/ variance
No free lunch theorem
model selection/ ensemble
MSE, MLE, Bayesian(MAP)
Supervised learning
Logistic regression
SVM
Decision tree
KNN
Unsupervised Learning
PCA
K-means
Challenge motivating of deep learning
Curse of Dimensionality
Local constancy and Smoothness regularization
Manifold Learning
Optimization
Gradient
AdaGrad (sqrt of sum of gradients before)
RMSprop (square of sum of updated gradients)
Adam (combine of momentum and RMSprop)
Momentum
Newton
Ensemble Method
Random Forest
Adaboost
XGBoost
Stack methods
Cost Function
distribution
MSE
cross entropy
statistic value of y
calculus of variations II
calculus of variations I
Decision Tree
Random Forest
Bootstrap samples
Random k split variable
Random split threshold
CART (Greedy)
Gini impurity
information gain
mean square error
