Please enable JavaScript.
Coggle requires JavaScript to display documents.
Communicate Model Insights (not all features are created equally (features…
Communicate Model Insights
useful info to understand the model vs useful info to an audience to make business decisions
6 types of info to be communicated in presentation
business problem
model quality metrics (confusion matrix)
areas where model struggles (potential for improvement through more data)
feature types especially interesting to management (insights into the business problem and unknowns uncovered during the modeling process)
most predictive features for model building
recommended business actions (to implement model or not, any business decisions to implement at various probability thresholds, and how will doing so change our predicitoin)
algorithms determine relationship between features and the target
Unlocking holdout
holdout sample provides an opportunity to check whether problems (with data/small sets may become innacurate) have occurred
best outcome is when the order of models doesn't change
2 best outcome is when the top model at least stays
if neither is true - then holdout sample scores are way lower than cross validation sample = reason for concern
Business Problem First
problem should be outlined and refined by this point
manipulations should be made
pre-processing and model quality metrics
procure the data, clean the data, carefully address issues
performance could deteriorate if the model is given new data
positive predictive value - values that were predicted to be positive or correct
false positive - not actually positive
true positive - actually is positive
also charts accuracy
areas where the model struggles
2 main types of data
internal data
external data - company would benefit from purchasing data,
most predictive features
at this point, unhelpful data will have been removed
not all features are created equally
features that need to be changed and therefore require a re-run of the models - target leak (defines a model with trained features)
features requiring further examination - who is going to ask follow up questions
immutable features - good for modeling but add no value to management
mutable features - features that management could potentially change, when manipulated they could change status
recommended business actions
implement the model - target candidates addressing the problem
institute a 1 month pilot program targeting 5% of target
institute a data-extraction-and-purchase pilot program to explore how the model could be improved and to what degree