Please enable JavaScript.
Coggle requires JavaScript to display documents.
Regularization and Overfitting (Ridge Regression (M not invertanle M +…
Regularization and Overfitting
Invertibility
Ridge Regression
M not invertanle
M + \(\lambda I\) = invertible
Same as J(\(\theta\)) + term
Intuition
Small weights
Large weights = complex model
Regularization term shrinks weights
Weights \(\propto\) overfitting
SMALL changes in x = LARGE changes in y
Leads to smaller models
\(\lambda\)
Hyperparamter
Nonlinear
\(\phi\) transformation
Multiple possibilities
Changes dimensions
Overfitting
Regularization
Trends
\(\lambda \to \infty\)
\(w \to 0\)
Don't trust data
\(\lambda \to 0\)
Weights don't change
Trust data
