Parametric vs Non-Parametric
Algorithms

Non-Parametric

Become more complex
with data increases

No parameter

Do not make strong assumptions
about the data

Has flexible number of parameters

Generalize to unseen data

Learning
Function

Can be summarized as learning a function f
such that Y=f(x) that map input x to output Y

Different algorithms make different assumptions or biases

Advantages

Disadvantages

Flexibility

  • Capable of fitting large number of functional form

Power

  • No/weak assumptions

Performance

  • Can result in high performance

Required a lot of training data

Slow in training

Overfitting

Parametric

Algorithms that simplify the function

Make strong assumption on data

Has a fixed number of parameters

2 Steps:

Select a form for the function

Learn the coefficients for the function
from the training data

Benefit

Limitation

Simple

  • Easy to understand & interpret result

Speed

  • Very fast to learn from data

Less data

  • Do not required much training data

Constrained

  • Highly constrained to the
    specified form

Limited Complexity

  • Suit for simple problem

Poor Fit

  • Unlikely to match
    the underlying mapping function