Please enable JavaScript.
Coggle requires JavaScript to display documents.
Larsen Chapter 17-18 (Chapter 17: Evaluate Model Performance (Intro (must…
Larsen Chapter 17-18
Chapter 17: Evaluate Model Performance
Intro
must make commitment to knowing all algorithms at technical level or knowing function of algorithms conceptually
key: ability to understand model's performance, understand model's business context
Sample Algorithm and Model
decision tree classifier
find most predictive feature (one that best explains the target) and place it at root of the tree
split the feature into two groups at the point of the feature where the two groups are as homogenous as possible
repeat step 2 for each new branch (box)
ROC Curve
Receiver Operating Characteristics Curve
measures
PPV; precision
TPR
F1
Accuracy
NPV
TNR; Specificity
FPR; Fallout
MCC
Using life chart for business decisions
enable drill down
Chapter 18: Comparing Model pairs
model comparison
allows selection of two models from leaderboard in addition to auto-selecting top model
Prioritizing Modeling Criteria and Selecting a Model
5 criteria
predictive accuracy
prediction speed
speed to build model
familiarity with model
insights