Please enable JavaScript.
Coggle requires JavaScript to display documents.
Ch. 17 Evaluate Model Performance (Introduction (Understanding and…
Ch. 17 Evaluate Model Performance
Introduction
Understanding and Learning
AutoML should improve user's understanding of the problem by visualizations of interactions between the features and target
know all algorithms at a technical level or knowing the function of these algorithms conceptually
FVE Binomial
provides sense of variance in dataset has been explained, equivalent to R^2-value
Sample Algorithm and Model
value in having strong general understanding of how algorithms work
more effective in model creation and validation
decision tree would be best to learn due to both its conceptual simplicity and its effectiveness
1)finds most predictive feature - place at root of tree
2) split feature into two groups at point of feature where the two groups are as homogenous as possible
3)repeat step 2 for each new branch (box)
ROC Curve
Receiver Operating Characteristics Curve screen
several central measures of model success exist beyond the original optimization metric, LogLoss
PPV - Positive Predictive Value
TPR - True Positive Rate
NPV - Negative Predictive Value
FPR - False Positive Rate
MCC - Matthews Correlation Coefficient
strong indicator when target of a business case is highly unbalanced
useful when PPV, TPR, and F1 are of little value if 99% of cases are positive and model classifies all cases as positive.
James Frainey
Jafr4672@colorado.edu