Please enable JavaScript.
Coggle requires JavaScript to display documents.
5720_3 (Mixtures of Gaussians (Maximum Likelihood Estimation of parameters…
5720_3
Mixtures of Gaussians
Complete and Incomplete Data
Maximum Likelihood Estimation of parameters
Gradient ascent (straightforward but slow)
K-means
not enable us to estimate covariance matrices
computationally much less complex
Fuzzy k means
Expectation-Maximization (EM) algorithm
expectation steps:solve for the responsibilities
maximization step:solve for the parameters
An Alternative View of EM
The EM Algorithm in General
Latent Variable Formulation
Dimensionality Reduction
Principal component analysis
Maximum variance formulation
Minimum-error formulation
Application
Compression
Data-preprocessing
Comparison with Fischer LDA
PCA for high dimensional data sets
Probabilistic PCA
Maximum Likelihood PCA
EM Algorithm for PCA
Independent component analysis
used in blind source separation
latent distribution is non-Gaussian
Lecture 11
Sequential data
i.i.d. = independent and identically distributed:poor
focus on the stationary case
Hidden Markov Models
Maximum likelihood for the HMM
EM algorithm
The forward-backward algorithm
the backward recursion for evaluation of the β variables.
the forward recursion for evaluation of the α variables
The Viterbi algorithm
most probable sequence
first-order Markov chain
higher-order Markov chains
Applications
Speech recognition
On-line handwriting recognition
Analysis of biological sequences such as proteins and DNA
Transition diagram
Transition Matrix
transition probabilities
lattice or trellis
emission probabilities
left-to-right HMM