Please enable JavaScript.
Coggle requires JavaScript to display documents.
Deep Learning (Bias and Variance (High Bias (underfitting)
Screenshot…
Deep Learning
-
-
Data sets
-
-
Dev and Test sets should come
from the same distributionTrain images are difficult to find in
large quantities, so use different
techniques to acquire them:
- generate data
- modify existing data
- web crawling
- etc.
However, Dev and Test data should come
from real-life usage (read data from users)
-
Parameter Initialization
Zeros
Does not break symmetry.
Each node of the network
learn the same function. Therefore,
the whole network works as linear unit.
Random
Breaks symmetry.
Make sure to initialize with
small numbers, otherwise gradient
descent would be slow and
weights (W) would most likely explode
He method
Works good for ReLu
Multiply weights by 