Please enable JavaScript.
Coggle requires JavaScript to display documents.
AI Week 4, Exactly 0.5 is known as the decision boundary, Re emergence of…
AI Week 4
Logistic Regression
-
Same concept as for linear regression but rather than finding a "best fit" line for the data points, we want to separate two categories with a boundary
If the classes are linearly separable, the
training error will be 0
The number of attributes of each data point affects how we represent our boundary (a single point, a line, a plane etc.)
-
We use a different cost function as mean square error isn't useful (produces a squiggly line so may get suck in local minima)
Instead we use the logistic cost function (see slides) which manipulates the result of the sigmoid function for each x,y pair and then averages the costs
Neural Networks
These are highly non-linear models with many free parameters; they can be used for regression and classification so generally replace logistic and linear regression since they're less practical
Model formulation: the main problem is making it specific to the problem at hand; we can think of a model as being made of several logistic regression units
Nodes are arranged in an input layer, output layer and any number of hidden layers (which typically apply functions - activation, sigmoid, ...)
-
Weights = parameters and Number of hidden units, choice of activation function, number of hidden layers = hyperparameters
-
Gradient descent learning algorithm: carried out through backpropagation (each iteration = one gradient descent step)
-
-
If x has d attributes, we have d+1 free parameters as components of w
-
The gradient descent step is similar but with a slightly different gradient vector due to differentiating a different calculation (see slides)
Nonlinear logistic regression: instead of linear function inside the exp in the sigmoid, we can use polynomial functions of the input attributes
-
-
Perceptron: A neural network with no hidden layers; equivalent to logistic regression if the activation function is sigmoid
