Chapter 20: Communicate Model Insights
introduction
knowledge on topics varies
if basic question, explain that it is something that contributes to explain the feature and target and offer to explain later
unlocking holdout (20.1)
first check for mistakes by releasing holdout
business problem first (20.2)
done by going to data robot to unlock holdout
do the top models remain the same
how has it changed with the data process
pre-process and quality metrics (20.3)
procuring/cleaning data and addressing issues
provide algorithms used
ex. removing expired patients
provide confusion matrix and explanation of what they mean
what measure means and why its better
remove data that is unnecessary in holdout sample
extract predicted probabilities
predict then add optional features then compute predictions
areas where the model struggles (20.4)
show why the model isnt perfect
internal data (need more?)
external data (need more?)
most predictive features (20.5)
develop story and explain
not all features are created equally (20.6)
features that need to be changed and require a re-run
target leak
why it was removed
recommend business actions
features requiring further examination
possible manager follow up questions names
immutable features
good for modeling but of no value to management
mutable features
could be changed by management
will the model lose efficacy?