Applied statistical modelling

Naive Bayes

Conditional independence

Independent random variables

Exchangeability

The first urn process

Polya urn

De Finetti theorem

Infinite exchangeability = conditional independence

Bayesian model

Predictive approach

Hierarchical models

Conjugacy

Conjugate prior

Gamma-Poisson

Beta-Binomial

Normal-Normal

Inversegamma-Normal

Monte Carlo integration

Monte Carlo Error

Non-conjugate models

Non-conjugate priors

Markov Chains

Features

Invariant distribution

LLN and CLT for Markov Chains

Autocorrelation function

2 types of Markov Chain

Metropolis-Hastings

Gibbs Sampler

Instrumental distribution proposal

Transition Kernel

Ergodic mean

Unbiased

Consistent

Irreducibility

Harris-recurrent

Reversibility

Convergence analysis

Burn-in

Geweke diagnostic

Gelman and Rubin diagnostic

2 problems

Convergence of the MC to stationary distribution

Dependence between elements of the Markov Chain

Thinning

Special case of a MH algorithm

Random sweep

Systematic sweep