Please enable JavaScript.
Coggle requires JavaScript to display documents.
Ch 20: Communicate Model Insights (6 pieces of info needed in presentation…
Ch 20: Communicate Model Insights
Info useful to understanding model
Info useful for making business decisions
6 pieces of info needed in presentation:
Areas where model struggles (more data or features)
Cross Validation and Holdout only show us problems with
Algorithm
- if we still can not solve predictability issue than we know that we have a
data issue
Internal v External
data
choice
Model quality Metrics (confusion matrix)
How data was prepped: cleaning, procuring, addressing issues (expired, not valid, etc)
What kind of models were run on this data (ex Logistic, decision tree, deep neural networks, etc)
Point out significance of confusion matrix: the most predictive outcome, most possibility for error (false positives)
Business Problem
Properly defining the business problem: this can Req. going back and forth with examining data and making sure we are even asking the right question > then proceed to answer that question with analysis
Most Predictive Features for Modeling
Look at model's predictive feature effect, to see which feature contributed most toward prediction
Feature types especially interesting to management (what insights to problems did feature engineering give us?)
Features that need to be changed lining up more with business req., therefore re-running model
Features that require further dissecting with a SME
Immutable features removes: features management cant change
Highlight mutable features: areas of improvement or changeable for management
Recommend business action (should model be implemented or not, if so what are the implications?)
3 recommendations:
Get more data to make better model
Run limited time and limited test subjects (high predictive) pilot program of model
Implement model
Answering Fundamental questions may not give enough time for meat of presentation
These matters can be discussed 1 on 1
Answered by: Generalizing relationships of features (independent variables) and target (dependent ) variable can predict unforeseen cases
Hold out sample: last sample to check model against (cross Validation vs Holdout)
Best option: both model ranking are matching
2nd best: top 3 are consistent in both
Option 3: holdout score < Cross Validation model's scores
something is most likely wrong with model's predictability