Parametric vs Non-Parametric
Algorithms
Non-Parametric
Become more complex
with data increases
No parameter
Do not make strong assumptions
about the data
Has flexible number of parameters
Generalize to unseen data
Learning
Function
Can be summarized as learning a function f
such that Y=f(x) that map input x to output Y
Different algorithms make different assumptions or biases
Advantages
Disadvantages
Flexibility
- Capable of fitting large number of functional form
Power
- No/weak assumptions
Performance
- Can result in high performance
Required a lot of training data
Slow in training
Overfitting
Parametric
Algorithms that simplify the function
Make strong assumption on data
Has a fixed number of parameters
2 Steps:
Select a form for the function
Learn the coefficients for the function
from the training data
Benefit
Limitation
Simple
- Easy to understand & interpret result
Speed
- Very fast to learn from data
Less data
- Do not required much training data
Constrained
- Highly constrained to the
specified form
Limited Complexity
- Suit for simple problem
Poor Fit
- Unlikely to match
the underlying mapping function