Please enable JavaScript.
Coggle requires JavaScript to display documents.
AI Week 6, Advantages, Disadvantages, Joint Probability Distribution: the…
AI Week 6
Probabilistic model: a mathematical description of an uncertain situation, with two main elements - sample space (set of all possible outcomes) and probability law (assigns a nonnegative number to each element of the sample space)
-
-
-
k-Nearest Neighbours: when trying to classify a new example, we choose a value for k and then choose the k nearest neighbours to the new example; the class with the most nearest neighbours will be chosen as the class of the new example
we start by normalising the numeric input attributes, which helps if there are different scales as it prevents some data skewing the result more than others
Advantages include training being simple and fast, we just have to store training data and we find the class of the new example based on most similar examples
Disadvantages include the space taken in memory to store all the data and running the algorithm may slow down when there are more examples or if there are lots of dimensions
Naïve Bayes
We can rearrange the product rule from before to get Bayes' Theorem; we use this to make predictions about an underlying process using the data produced as a knowledge base
Given an output class, C, and input values, a, we calculate P(C|a) using Bayes Theorem and use the max P(C|a) of all classes as our prediction.
We can replace part of the equation with a constant parameter as it remains constant for each C and is seen as a normalisation constant
We can easily deal with multiple independent variables by using ai to represent the value where a1 to ai are each independent variables we take into account (see notes)
Problems: Scaling is a problem because we need a table of size 2^n and processing also requires this complexity. Missing values lead to multiplying later values by 0 (missing if they're not observed in our training set)
Categorical Independent Variables: We make the assumption that each independent variable is conditionally independent of eachother, given the output which prevents exponential growth - e.g. P(x1| x2,y) = P(x1,Y) SEE NOTES
-
Since this causes inaccuracies (as we're not using one of the variables), we use laplace smoothing to calculate all the values that use ai; we don't need to for P(C)
Laplace Smoothing: adding 1 to the frequency of all elements and using the updated tables to calculate P(ai|C)
Numerical Independent Variables: We assume probability is drawn from a probability distribution (as probability of a single value is 0) so create a PDF for each outcome value, calculating mean and variance
We can use mean and variance to create a parameter table and use the formula in the notes to find P(x)
PMF (Probability Mass Function): captures the probabilities of the result being one of the possible variables a discrete random variable can take
Unconditional Probability Distribution: gives us probabilities under the assumption we know nothing else about the problem
Conditional Probability Distribution: gives us the probability with additional information, such as the first roll being a 3 on a dice
PDF (Probability Density Function): captures the probabilities of a continuous variable in terms of a nonnegative function; since the possible values are infinite, we can't talk about the probability of just 1 and instead find the negative likelihood.
-
-
Joint Probability Distribution: the probability distribution associated to all combinations of the values of two or more random variables; indicated with commas and calculated with the product rule
here, we also discuss mean, variance and standard deviation
It's known as Naïve Bayes because we assume that all variables are independent and contribute an equal weight
A nonparametric model is a model that cannot be characterised by a
bounded set of parameters (e.g. training set grows over time)
-
ORDINAL INPUT ATTRIBUTES: we convert to numeric values (e.g. yes=1, no=0)
CATEGORICAL INPUT ATTRIBUTES: if the values are equal, their difference is 0; otherwise, its 1