Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 20: Communicate Model Insights (20.3 Pre-processing and Model…
Chapter 20: Communicate Model Insights
Intro
5 types of info that should be communicated during a presentation:
Areas where a model struggles (potential for improvement through more data––features & cases)
Most predictive features for model building
Model quality metrics (confusion matrix)
Feature types especially interesting to management (e.g., insights into the business problem and unknowns uncovered during the modeling process)
Business Problem
Recommended business actions
20.1 Unlocking Holdout
check that we haven’t made any mistakes in the model creation process by releasing the holdout data
holdout sample allows us to check whether such problems may have occurred
the best outcome would be one in which the order of models did not change between the two sorts.
20.2 Business Problem First
problem should have guided your work continually
problem also should have been refined
also considered recreating decisions
20.3 Pre-processing and Model Quality Metrics
model metrics from the confusion matrix were combined to better understand the model performance characteristics
Slide 3 might contain a quick overview, explaining the process for procuring data, cleaning that data, carefully addressing issues
For slide 4, to address model quality metrics, let us start with the confusion matrix for our particular example and annotate it a bit for our audience
For slide 5, we would normally use the confusion matrix screen with a different probability distribution threshold to identify a probability above which a small set of patients would be highly likely to be readmitted
In short, this slide focused on convincing management that the model would save money for the organization
20.4 Areas where model struggles
If management is convinced that there is value in this analysis for the patients that the model assigns high probabilities of readmission, and the proposed pilot and educational and support interventions are successes with the current data, we believe that arguing for more data is reasonable.
2 types of data:
Internal data
External data.
For the sixth slide, we are arguing for a pilot project, because throughout the discussion of this dataset, we have addressed that, quite simply, it is not a very predictive dataset
20.5 Most predictive features
slide 7: When explaining a model, directional results tend to make intuitive sense
Be ready to develop a story
20.6 Not All Features are Created Equally
4 kinds of features to consider before going into a management presentation:
Immutable features
Mutable features
Features requiring further examination
Features that need to be changed and therefore requiring a re-run of the models
20.7 Recommended Business Actions
3 recommendations:
Institute a one-month pilot program targeting five percent of patients discharged to home
Institute a data-extraction-and-purchase pilot program to explore how much more predictive the patient readmission model can become
Implement the model to find the 62% of patients most likely to be readmitted and therefore, implement educational and support programs.