Please enable JavaScript.
Coggle requires JavaScript to display documents.
Training Deep Neural Nets (:star: 11.3 Faster Optimizers (11.3.1 Momentum…
Training Deep Neural Nets
11.1 Vanishing/Exploding Gradients Problems
:star: 11.1.3 Batch Normalization
11.1.1 Xavier and He Initialization
11.1.4 Gradient Clipping
11.1.2 Nonsaturating Activation Functions
11.2 Reusing Pretrained Layers
:star: 11.3 Faster Optimizers
11.3.1 Momentum optimization
11.3.2 Nesterov Accelerated Gradient
11.3.3 AdaGrad
11.3.4 RMSProp
:star: 11.3.5 Adam Optimization
:star: 11.3.6 Learning Rate Scheduling
11.4 Avoiding Overfitting Through Regularization
11.4.1 Early Stopping
11.4.2 l1 and l2 Regularization
:star: 11.4.3 Dropout
11.4.4 Max-Norm Regularization
11.4.5 Data Augmentation
11.5 Practical Guidelines