Chapter 20. Communicate Model Insights (Chapter intro (There are six types…
Chapter 20. Communicate Model Insights
It is important to distinguish between information useful for understanding the model and information useful to an audience for making business decisions.
Understanding the model is important for the data scientist creating and modifying the model, but a seldom few model details are relevant to the final stakeholder audience.
Presentations should be adjusted based on personal experiences with the audience, as well as an understanding of the business context.
There are six types of information that should be communicated during a presentation:
Model quality metrics (confusion matrix)
Areas where a model struggles (potential for improvement through more data––features & cases)
Most predictive features for model building
Feature types especially interesting to management (e.g., insights into the business problem and unknowns uncovered during the modeling process)
Recommended business actions (i.e., to implement model or not, any business decisions to implement at various probability thresholds, and how will doing so change practice?)
20.1 Unlocking Holdout
it is first important to check that there haven’t been any mistakes made in the model
This is done by releasing the holdout data.
20.3 Pre-processing and Model Quality Metrics
Explain that many different high quality algorithms were run on the data, and feel free to point out examples of these, such as logistic regression and deep-learning neural networks.
20.2 Business Problem First
During this analysis, some details became clear about decisions made by the physicians in the study, including discharge decisions.
20.4 Areas where the model struggles
If management is convinced that there is value in this analysis for the patients that the model assigns high probabilities of readmission, and the proposed pilot and educational and support interventions are successes with the current data,arguing for more data then becomes much more reasonable.
two main types of data
20.5 Most predictive features
When explaining a model, directional results tend to make intuitive sense
20.6 Not All Features are Created Equally
Often information that helps a model make high-quality decisions is not helpful in changing practice.
There are four kinds of features to consider before going into a management presentation:
Features that need to be changed and therefore require a re run of the models
Features requiring further examination