Please enable JavaScript.
Coggle requires JavaScript to display documents.
ch 2 ML Simple ML Classification Algorithms (Introduce NumPy (random…
ch 2 ML Simple ML Classification Algorithms
Perceptron Learning Rule
compares true class labels to predicted class labels
Introduce One Versus All Technique
Introduce NumPy
random.RandomState(seed random)
array - similar to python lists, vectorizes arithmetic operations; meaning an operation is applied to all elements (?) Faster.
x=where(bool,false,true) sets array of x true/false
meshgrid(x1,...xn) Return coordinate matrices from coordinate vectors
np.arange(min,max,resolution) Return evenly spaced values within a given interval
Demonstrate scatter plot, epochs line graph, decision boundary graph
plot_dec
Adaline Neural Network
compares true class labels with linear activation function’s continuous valued output to model error
Introduce problem of selecting eta, learning rate
Gradient Descent
feature scaling through standardization
shifts mean of each feature so that it centers at zero with std dev of 1. Reduces number of training epochs.
Stochastic gradient descent
update weights incrementally for each training sample, rather than based on sum of accumulated errors
used in online learning, as new data arrives, and to adjust for local minimum problems
Techniques and Libraries
Problems
Core concepts
Graphing and visualization concepts
Introduce Matplotlib
pyplot module
scatter(x,y, color, marker, label) creates a scatter plot
xlabel, ylabel,legend(loc=''),show to display
colors module
cmap=ListedColormap(colorslist[:len(np.unique(y))]) to create a list of colors equal to length y
Introduce Pandas
df=read_csv(html, header=None) pull dataset from csv file, specify if header; can also read_csv(local file)
df.iloc[row,column].values returns values
ch 3 scikit-learn Classifiers
Problem of choosing a classification algorithm
No single classifier works best across scenarios, so try several
Avoid overfitting
Tune via regularization with reg param lambda
Approach to selecting a classifier
Select features, collect training data
Choose performance metric
Choose classifier and algorithm
Evaluate performance
Tune the algorithm
introduce scikit-learn, which conveniently has sample data sets
model_selection module
train_test_split method to choose split ratio
preprocessing module
StandardScalar medthod which does our feature scaling for us
linear_model module
Perceptron class
fit(x training, y training) does our PLR
predict(x test) predicts y dataset
LogisticRegression
metrics module
tree module
DecisionTreeClassifer
ensemble
RandomForestClassifier, good performance and ease of use; don’t have to worry about choosing good hyperparameter values
neightbors
KNeighborsClassifier non parametric, adapts as we collect data, complexity grows with number of samples.
Numpy
x.ravel() return a contiguous flattened array.
matplotlib
.contourf(x1,x2,x3, alpha) draw filled contours.
Logistic Regression
odds ratio: p/(1-p). approaches infinity as p->1
logit function: log(p/(1-p))
logistic sigmoid function: 1/(1+e^-z)
Performs well on linearly separable classes, and is easy to implement. Widely used.
implement with sklearn.linear_model.LogisticRegression
Kernel Trick: Find Separating Hyperplanes
mapping function phi, train SVM model to new space
Common Kernel: Radial Basis Function/Gaussian kernal
with parameter gamma to optimize cutoff
Decision Tree Learning
Good for interpretability
split data on feature resulting in largest information gain
prune trees to reduce overfitting
Random Forest DTree Learning
Trade some interpretability for performance
Nearest Neighbors
lazy, doesn’t learn, simply memorizes training data.
Algorithm grows with the size of data presented; non parametric = not fixed set of parameters. Kernel SVM and Trees are also non parametric.