ST2334 Probability and Statistics

ST2334 Probability and Statistics

Descriptive
Summarizing/Describing data

Inferential
-Drawing conclusions on population based on sample

Variables

Categorical

Numerical
-Discrete
-Continuous

Sampling

Probability

Non-probability

Simple Random

Systematic

Cluster
Clusters tend to be similar to one another

Stratified
Units in a strata share common traits

Quota

Convenience

Judgement

Sample Variance Formula
w/o computational formula
$s^2=\frac{1}{n-1}\Sigma^{n}_{i=1}(x_i - \bar{x})^2$
(Computational Ver.)
$\frac{1}{n-1}[\Sigma^{n}_{i=1}x_i^2-\frac{(\Sigma_{i=1}^{n}x_i)^2}{n}]$
StdDev = s

Skew
Left = Mean < Median
Right = Mean > Median

Coefficient of Variation
100%*$\frac{stdDev}{|mean|}$

5-number summary:
Min,Q1,Median,Q3,Max

Interquartile Range
IQR=Q3-Q1

Probability
P(A ∪ B) = P(A)+P(B)

Counting Methods

Generalized Basic Principle of Counting
Factorial (n!) [Arranging n obj]
Permutations ((n!)/(n-r)!) [Arranging r out of n]
Combinations ((n!)/[r!(n-r)!])

Conditional Prob.
P(B|A) = $\frac{P(A∩B)}{P(A)}$
P(A∩B) = P(A) P(B|A)
Inverse Prob.
P(A|B) = $\frac{P(A∩B)}{P(B)}$

Independent Prob iff
P(A∩B) = P(A) P(B)
Mutual Exclusive
P(A∩B) = 0

Bayes' Theorem
P(A|B) = $\frac{P(A) * P(B|A)}{P(B)}$,
where P(B) [the denominator]
= Total Prob. of B
= $\Sigma^n_{i=1}P(A_i) P(B|A_i)$

Random Variables

Discrete
CDF
$F(x) = P(X ≤ x) = \Sigma_{k=1}^{x} P(X=k)$

Expectation EV = Mean value of random variable
$E(X)=\Sigma_{x_i}x_iP(X=x_i)$
Properties:
E (a+bX) = a + bE(X)
E(X+Y) = E(X)+ E(Y)
E( f(x) ) = $\Sigma_{x_i}f(x) P(X=x_i)$

Variance & stdDev
var X
= $E([X-E(X)]^2)$
= $\Sigma_{x_i} (x_i - u)^2 P(X=x_i)$
= $E(X^2) - [E(X)]^2$

Variance Property: var (a+bX) = b^2 var X

StdDev of random variable X
= $\sigma = SD(X) = \sqrt{var X}$

(No. of successess in n trials)
Binomial Random Var.
X~B(NoOfEvents[n], ProbOfSuccess[p])
P(X=x)
= $\binom{n}{x}p^x (1-p)^{n-x}$
, for x=1,2,..,n
$E(X) = np$
$var X = np(1-p)$

#Large n, Small p
Let $X_n$~B($n,\frac{\lambda}{n}$),
Y~Poisson($\lambda$),
$lim_{n->∞} P(X_n=k) = P(Y=k)$
May be approxed by Poisson #

Let X be discrete,
$ X\approx N(np,np(1-p)) $
$ P(X≤x)\approx \Phi (\frac{x-np}{\sqrt{np(1-p)}}) $
May be approxed by Normal #
*Continuity Correction when approx. norm.
Represent continuous ranVar k with
k-0.5 to k+0.5
-0.5 from the discrete ranVar., keep arrow sign

(No. of B.Trials needed before success)
Geometric Random Variables
$P(X=k) = (1-p)^{k-1}p$, for k=1,2,3,...
$E(X) = \frac{1}{p}$
$var X = \frac{1-p}{p^2}$

CDF:
$\Sigma_{k=1}^{x} P(X=k)$
=$\Sigma_{k=1}^{x} (1-p)^{k-1} p$
=$1-(1-p)^x$

(No. of events in fixed period)
Poisson Random Variables
$\lambda$ is the "rate" and should be scaled accordingly
$P(X=k)=\frac{e^{-\lambda}\lambda^{k}}{k!}$
$E(X) = \lambda$
$var X = \lambda$

(Bernoulli - Independent, Identical Trials with 2 outcomes)
Bernoulli Random Variable
P(X=x):
p, when x=1,
1-p, when x=0
Derived:
E(X) = p,
var X = p(1-p)

Continuous
P(X=x) = 0, for any x

P(a<X≤b) = $\int_{a}^{b}f_x(x) dx$
$\int_{-∞}^{∞} f(x) dx = 1$

CDF
$F(x) = \int_{-∞}^{x} f(t) dt$
P(a<X≤b) = F(b) - F(a)

Expectation
$\mu = E(g(X)) = \int_{-∞}^{∞}g(x) . f(x)dx$

Variance
$var(X) = E[(X - E(X))^2] = \int_{-∞}^{∞} (x-\mu)^2 f(x) dx $
OR
$var(X) = E(X^2) - [E(X)]^2 = \int_{-∞}^{∞} x^2 f(x) dx - \mu^2$

Normal/Gauss Distri. (X ~ N($\mu, \sigma^2$))
$ f_x(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-(x-\mu)^2 / (2\sigma^2)} $
$ E(X) = \mu, var(X) = \sigma^2 $

Uniform Distri.
(X~ U(a,b))
$ f_x(x) = \begin{cases} 1/(b-a) , a < x < b \\ 0, otherwise \end{cases} $
$ E(X) = (a+b)/2 $
$ var(X) = \frac{(b-a)^2}{12} $

CDF
$ F_x(x) = \begin{cases} \frac{1}{(b-a)} , a < x < b \\ 0, otherwise \\ 1, if x ≤ b \\ \end{cases} $

Exponential Distri.
(X~ Exp($\lambda$))
$ f_x(x) = \begin{cases} \lambda e^{-\lambda x}, if x≥ 0 \\ 0, if x<0 \end{cases} $
$ E(X) = \frac{1}{\lambda} $
$ var(X) = \frac{1}{\lambda^2} $

CDF
$ F_x(x) = \begin{cases} 1- e^{-\lambda x}, x>0 \\ 0, x≤0 \\ \end{cases} $

Memoryless Property
P(X > s+t | X > s)
= P(X>t), for s,t>0

Standard Normal Distri.
(Z~N(0,1))
Same, but with $\mu = 0$ and $\sigma = 1$

For Y~N($\mu, \sigma^2)$
$ P(a < Y ≤ b)=Φ(\frac{b-\mu}{\sigma}) - Φ(\frac{a-\mu}{\sigma}) $

Properties:
P(Z≥0) = P(Z≤0) = 0.5
P(Z≤x) = 1-P(Z>x)
P(Z≤ -x) = P(Z≥x)
-Z~N(0,1)

If Y ~ N $(\mu, \sigma^2), then X = \frac{Y-\mu}{\sigma}$~N(0,1)
If X~N(0,1), then Y = aX+b ~ N(b,$a^2$)

Chebyshev's Inequality
Used when distri. unknown to estimate prob., given mean and variance

If random variable X has mean and SD, prob. of getting val. which deviates from $\mu$ by at least $k\sigma$ is at most $\frac{1}{k^2}$

Formula (sub in k accordingly):
$ P(|X-\mu| ≥ k\sigma) ≤ \frac{1}{k^2} $

Quantile (Z-score table)
The qth quantile of the random variable X is the number $z_q$ that satisfies
P(X ≤ $z_q$) = q.

Sampling and Sampling Distributions
Sampling Distri. of $\bar{X}$
$ E(\bar{X)} = \mu, var(\bar{X}) = \frac{{\sigma^2}}{n} $
Law of Large Numbers
$P(|bar{X} - \mu| > \epsilon)$ -> 0 as n -> inf
Central Limit Theorem
For large n, sum and mean of random samples follow normal distribution closely
$ \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} $~ N(0, 1)

t-distribution with t(ail)-table
If $\sigma$ isn't known, it can be estimated by the sample StdDev
$ T = \frac{\bar{X} - \mu}{S / \sqrt{n}} $
is a random variable having the t-distribution with parameter v=n-1, where v is the degree of freedom. When n>=30, can replace by N(0,1)

(Note: Chi2 is only applicable for normal populations)
$\chi^2$ distri. with $\chi^2 $table
If $X_i$ is normal,
$ \chi^2 = \frac{(n-1)S^2}{\sigma^2} = \frac{\Sigma_{i=1}^{n}(X_i - \bar{X})^2}{\sigma^2} $
is a random variable having the $\chi^2$ distribution with param. v=n-1