Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 7: What Makes a Good Model? (Expected Value (calculation…
Chapter 7: What Makes a Good Model?
Evaluating Classifiers
Methods
Plain Accuracy
evaluates model
based on
accuracy of classification
is simple
because
easy to measure
but usually
too simple
for business problems
measured by
comparing
correct decisions
total decisions
ultimately
do not use
to determine model qualty
Confusion Matrix
compares
actual class values
predicted class values
shows
if one class
is being confused for another
identifies
false positives
#
occur when
something is classified as positive
but is actually
negative
false negatives
#
occur when
something is classified as negative
but is actually
positive
"Good" Models
want to
ensure
model addresses
original problem
often overlooked
consider
acceptable outcomes
before modeling data
can't be
too "good"
leads to
overfitting data
too specific
Qualifiers
uneven costs and benefits
classification accuracy
does not distinguish
between
false positives
false negatives
unbalanced classes
occur
when one class
is rare
doesn't have
equal distribution
causes
data to
disregard uneven distribution
Expected Value
calculation
enumerates
all possible outcomes
weighs
different outcomes
on their
likelyhood of occurance
shows
how the model
will be used
identifies
error rates
applies
costs and benefits
#
to the model