Please enable JavaScript.
Coggle requires JavaScript to display documents.
Bayesian Statistic (Probability distribution (Constructing a likelihood…
Bayesian Statistic
Probability distribution
Multivariate distriutions
Conditional distributions
\[ f(x{\mid}y)= \frac {f(x,y)} {f(y)} \]
Marginal distribution
\[ f(y)= \int_{x \in S} f(x,y)dx \]
Spectrum
factor analysis
structural equation modeling with latent variables
simultaneous equation modeling
A density that can integrate to 1 with an appropriate normalizing constant is called a “proper” density function.
continuous
\(\mu_x=\int_{x\in S}xf(x)dx\)
\({\sigma_x}^2=\int_{x \in S}(x-\mu_x)^2 \times f(x) dx\)
mass function
discrete
\(\mu_x=\displaystyle\sum_{x \in S}x \times p(x)\)
\({\sigma_x}^2=\displaystyle\sum_{x \in S}(x-\mu_x)^2p(x)\)
density function
Densities that involve more than one random variable are called joint densities
Constructing a likelihood function
\(f(X{\mid}\theta){\equiv}L(\theta \mid x)=\displaystyle\prod_{i=1}^n f(x_i \mid \theta)\)
Estimate the parameter \(\theta \), given the data have been observed
\(LL(\theta \mid X)=\displaystyle\sum_{i=1}^n ln(f(x_i \mid \theta))\)
standard errors
\(I(\theta)^{-1}=(-E(\frac{\partial^2 LL}{\partial \theta \partial \theta^T}))^{-1}\)
information matrix
The square root of the diagonal elements of the matrix are the parameter standard errors
rate of curvature of the curve
uniform distribution
Random samples from other distributions are generally simulated from draws from uniform distributions
They are commonly used as priors on parameters when little or no information exists to construct a more informative prior
Prior distribution
Beta distribution
Subjective and non-informative
Inverse gamma distribution
Dirichlet distribution
Wishart and inverse Wishart
Generalization of gamma and inverse gamma
Typical Bayesian steps
Establish a model and obtain a posterior distribution for the parameters of interest
Generate samples from the posterior distribution
Use discrete formulas applied to the samples from the posterior distribution to summarize our knowledge of the parameters
Computing integrals
Sampling Methods
Gibbs Sampler
Gibbs sampling
\[ Y_{0}^{'}, X_{0}^{'}, Y_{1}^{'}, X_{1}^{'}, Y_{2}^{'}, X_{2}^{'}, ..., Y_{k}^{'}, X_{k}^{'} \]
Can be exploited
in a variety ways
\[ X_{j}^{'} \sim f(x \mid Y_{j}^{'} = y_{j}^{'})\\ Y_{j+1}^{'} \sim f(y \mid X_{j}^{'} = x_{j}^{'}) \]
\[ \lim_{m \to \infty} \frac {1}{m} \displaystyle\sum_{i=1}^{m} X_i = \int_{-\infty}^\infty xf(x)dx = EX \]
Markov Chain Monte Carlo
Adaptive Rejection Sampling
Slice Sampler
Independence Sampler
Hamiltonian Monte Carlo Sampler
Inversion
Rejection
Limitations
Finding \( m \times g(x)\) may not be easy
It may not be efficient
Metropolis hastings
It works with multivariate distributions
basic steps
Establish starting values
S
for the parameter: \(\theta^{j=0}=S\). Set \(j=1\).
MLE
Draw a “candidate” parameter, \(\theta^c\) from a “proposal density,” \({\alpha}(.)\).
Compute the ratio \(R=\frac{f(\theta^c)\alpha(\theta^{j-1}{\mid}{\theta^c})}{f(\theta^{j-1})\alpha(\theta^c{\mid}\theta^{j-1})}\)
Compare \(R\) with a \(U(0,1)\) random draw
u
. If \(R>u\), then set \(\theta^j=\theta^{j-1}\).
Set \(j=j+1\) and return to step 2 until enough draws are obtained.
all Gibbs sampling is MH sampling, but not all MH sampling is Gibbs sampling
Approximation
Quadrature
Taylor series expansions around the mode of the log-posterior distribution
Posterior distribution
\[ p({\theta}{\mid}{data})= \frac {L({\theta}){\pi}({\theta})} {{\int}L({\theta}){\pi}({\theta})d{\theta}} \]
\[ p({\theta}{\mid}{data})∝ {L({\theta}){\pi}({\theta})} \]
Advantages & disadvantages
Advantages
Disadvantages
Doesn't tell you how to select a prior
Posterior distribution
heavily influenced by the priors
Degree of belief
Gamerman
Conjugate prior