Please enable JavaScript.
Coggle requires JavaScript to display documents.
ML for Dimesional Reduction I (Ashley Villar) - Coggle Diagram
ML for Dimesional Reduction I (Ashley Villar)
Fitting a model using MOO
Model
Objective fucntion
metric that you will choose to quantify how well the model fits the data
Optimization Method
Removing information can be an effective way to remove dimensionality
We want to create a low-dimensional representation without directly fitting a physical model
One simple way is to break down data into
basis vectors
Keywords to search
basis vectors
PCA
PCA (Principal Components Analysis) steps
Find the eigenvectors of the dataset
sort the eigenvectors by explained variance
Project the data onto each basis, tracking the weights
In 3 dimensions, it can be a plane that the data will be projected on that
Power iteration method
When we have too many dimensions and we want less dimensions
We choose the most important dimensions
Steps
Missed or not understood
PCA power iteration method steps in page 22
t-distributed stochastic neighbor embedding method
(t-SNE)
Neighbor embedding: quantify which observations are similar
How we do that generally?
We define a distance metric (e.g. Euclidean distance)
Think of this distance as being proportional to the chance that observations are "neighbors"
We will have a free parameter to tune (distance)
in order to minimize the difference between these distributions
Kullback-Leibler divergence
while PCA is deterministic, t-SNE is a stochastic method
Page 46 shows a good table for pros and cons of each method