Please enable JavaScript.
Coggle requires JavaScript to display documents.
12 The Analysis of Variance (One-way layout (Normal Theory; the F test…
12 The Analysis of Variance
One-way layout
Normal Theory; the F test
Our model is that the observations are corrupted by random errors and that
the error in one observation is independent
of
the errors in the other observations
:\[Y_{ij}=\mu+\alpha_{i}+\varepsilon_{ij}\]
\(\mu\) overall mean
\(\alpha_i\) the
differential effect
of the
i
th treatment
normalized: \(\sum_{i=1}^{I}\alpha_{i}=0\)
\(\varepsilon_{ij}\) the
random error
in the
i
th treatment's
j
th observation
The errors are assumed to be independent ~ N(0, \(\sigma^2\))
The expected response to the
i
th treatment is \(E(Y_{ij})=\mu+\alpha_i\)
Thus, if \(\alpha_i=0\) for all \(i=1,...,I\), all treatments have the same expected response, and, in general, \(\alpha_i-\alpha_j\) is the difference between the expected values under treatments \(i\) and \(j\).
The analysis of variance is
based on the following identity
:\[\sum_{i=1}^{I}\sum_{j=1}^{J}(Y_{ij}-\overline{Y}_{..})^{2}=\sum_{i=1}^{I}\sum_{j=1}^{J}(Y_{ij}-\overline{Y}_{i.})^{2}+J\sum_{i=1}^{I}(\overline{Y}_{i.}-\overline{Y}_{..})^{2}\]
\[\overline{Y}_{i.}=\frac{1}{J}\sum_{j=1}^{\textrm{J}}Y_{ij}\]
\[\overline{Y}_{..}=\frac{1}{IJ}\sum_{i=1}^{I}\sum_{j=1}^{J}Y_{ij}\]
#
kan användas för att estimera \(\sigma^2\); estimatet blir ju då\[s_{p}^{2}=\frac{SS_{W}}{I(J-1)}\]
#
kan också estimera \(\sigma^2\) i fallet när alla \(\alpha_i=0\), dvs under nollhypotesen. Detta ger estimatet:\[\frac{SS_{B}}{I-1}\]
I det fallet borde alltså de två estimaten vara ungefär lika, \[\frac{SS_{W}}{I(J-1)}\approx\frac{SS_{B}}{I-1}\]
Eftersom vi vet
nollfördelningen
av kvoten \(SS_W/SS_B\) så är ju det en användbar teststatistika!
experimental design in which
independent
measures are made
under each of several treatments
(dvs generalisering av teknikerna för att jämföra
två oberoende samples
i kapitel 11)
\(I\)
independent samples
of equal size \(J\)
\(Y_{ij}\) = the
i
th treatment's
j
th observation
\(H_0\): all \(I\) treatments have the same effect
\(H_1\): there are systematic differences
What about when the numbers of observations under the various treatments are
not necessarily equal
?
The Problem of Multiple Comparisons
Ett sätt skulle ju vara att jämföra alla par med t-testet,
men problemet är samlingen av jämförelser skulle ha en mycket högre Typ-1-fel-rate än varje tests enskilda rate \(\alpha\)
Two solutions
Tukey's method
The idea is that all the differences are less than some number if and only if the largest difference is. -> a set of confidence intervals that hold
simultaneously for all differences \(\mu_{u}-\mu_{v}\)
:
Studentized range distribution \(SR(k, df)\) has two parameters: the
number of samples
\(k\), and the number
of
degrees of freedom used in the variance estimate
\(s_p^2\)
If \(I\) independent samples \((Y_{i1},...,Y_{iJ})\) taken from \(N(\mu_i, \sigma^2)\) have the same size \(J\), then the sample means \(\overline{Y}_{i.}\sim N(\mu_{i},\frac{\sigma^{2}}{J})\) are independent and\[\frac{\sqrt{J}}{s_{p}}\max_{u,v}\left|\overline{Y}_{u.}-\overline{Y}_{v.}-(\mu_{u}-\mu_{v})\right|\sim SR(I,I(J-1))\]
the upper 100\(\alpha\) percentage point of the distribution is denoted by \(q_{\,I,\,I(J-1)}(\alpha)\)
The Bonferroni Method
Warning: \({I}\choose{2}\) pairwise Anova comparisons
are not independent as required
by Bonferroni method
indeed, assuming the null hypothesis is true, the number of positive results is \(X\sim\textrm{Bin}(k,\frac{\alpha}{k})\), and due to independence \(P(X\geq1|H_{0})=1-(1-\frac{\alpha}{k})^{k}\approx\alpha\) for small values of \(\alpha\).
Think of \(k\) independent replications of a statistical test. The overall result is positive if we get at least one positive result
among these \(k\) tests. The overall significance level α is obtained,
if each single test is performed at significance level α/k
:
Simultaneous \(100(1-\alpha)%\) formula for \(I\choose 2\) pairwise differences \((\alpha_u-\alpha_v)\):\[(\overline{Y}_{u.}-\overline{Y}_{v.})\pm t_{I(J-1)}\left(\frac{\alpha}{I(I-1)}\right)\cdot s_{p}\sqrt{\frac{2}{J}}\]
Flexibility of the formula: works for different sample sizes as well after replacing \(\sqrt{\frac{2}{J}}\) by \(\sqrt{\frac{1}{J_{u}}+\frac{1}{J_{v}}}\)
The idea is very simple. If \(k\) null hypotheses are to be tested, a desired overall type I error rate of at most \(\alpha\) can be guaranteed by testing each null hypothesis at level \(\alpha/k\).
Equivalently, if \(k\) confidence intervals are each formed to have confidence level \(100(1-\alpha/k)%\), they all hold simultaneously with confidence level at least \(100(1-\alpha)%\).
The method is simple and versatile and, although crude, gives surprisingly good results if \(k\) is not too large.
A Nonparametric Method - The Kruskal-Wallis Test
\(R_{ij}\) = the rank of \(Y_{ij}\) in the combined sample
\[\overline{R}_{i.}=\frac{1}{J_{i}}\sum_{j=1}^{J_{i}}R_{\textrm{ij}}\]
\[\overline{R}_{..}=\frac{1}{N}\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}R_{ij}=\frac{N+1}{2}\]
As in
#
, let\[SS_{B}=\sum_{i=1}^{I}J_{i}(\overline{R}_{i.}-\overline{R}_{..})^{2}\]be a measure of the dispersion of the \(\overline{R}_{i.}\).
\(SS_B\) may be used to test \(H_0\): the prob. distr. generating the observations under the various treatments are
identical
. The larger \(SS_B\) is, the stronger the evidence against the null hypothesis.
Two-way layout
experimental design involving
two factors
, each at two or more levels
differential effect for a day
= mean for that day - overall mean
Additive model
\[\hat{Y}_{ij}=\hat{\mu}+\hat{\alpha}_{i}+\hat{\beta}_{j}\]
\(\hat{Y}_{ij}\) = the predicted/fitted values of \(Y_{ij}\)
\(Y_{ij}\) observed values
\(Y_{ij}-\hat{Y}_{ij}\) are the
resíduals
from the additive model
Interactions
can be incorporated in the model to make the data fit perfectly: the residuals are\[\hat{\delta}_{ij}=Y_{ij}-\hat{Y}_{ij}=Y_{ij}-\overline{Y}_{i.}-\overline{Y}_{.j}+\overline{Y}_{..}\]
Assume K independent observations are taken from each of the I*J combinations
ex. I drugs, J genders
ex. I menydagar, J spisar (figur)
Normal theory for the Two-way layout
Assume K>1 observations per cell
A design with an equal number of observations per cell is called
balanced
\(Y_{ijk}\) = cell ij:s k:te observation
Den statistiska modellen är\[Y_{ijk}=\mu+\alpha_{i}+\beta_{j}+\delta_{ij}+\varepsilon_{ijk}\]
\(\varepsilon_{ijk}\) assumed to be \(N(0,\sigma^2)\)
and
independent
\[\sum_{i=1}^{I}\alpha_{i}=0\]
\[\sum_{j=1}^{J}\beta_{j}=0\]
\[\sum_{i=1}^{I}\delta_{ij}=\sum_{j=1}^{J}\delta_{ij}=0\]
As expected, the mle are:
\(\hat{\mu}=\overline{Y}_{...}\)
\(\hat{\alpha}_{i}=\overline{Y}_{i..}-\overline{Y}_{...}\)
\(\hat{\beta}_{j}=\overline{Y}_{.j.}-\overline{Y}_{...}\)
\(\hat{\delta}_{ij}=\overline{Y}_{ij.}-\overline{Y}_{i..}-\overline{Y}_{.j.}+\overline{Y}_{...}\)
\[SS_{TOT}=SS_{\textrm{A}}+SS_{B}+SS_{AB}+SS_{\textrm{E}}\]
\[SS_{A}=JK\sum_{i=1}^{I}(\overline{Y}_{i..}-\overline{Y}_{...})^{2}\]
\[SS_{\textrm{B}}=IK\sum_{j=1}^{J}(\overline{Y}_{.j.}-\overline{Y}_{...})^{2}\]
\[SS_{AB}=K\sum_{i=1}^{I}\sum_{j=1}^{J}(\overline{Y}_{ij.}-\overline{Y}_{i..}-\overline{Y}_{.j.}+\overline{Y}_{...})^{2}\]
\[SS_{\textrm{E}}=\sum_{i=1}^{\textrm{I}}\sum_{j=1}^{J}\sum_{k=1}^{K}(Y_{ijk}-\overline{Y}_{ij.})^{2}\]
\[SS_{TOT}=\sum_{i=1}^{I}\sum_{j=1}^{J}\sum_{k=1}^{K}(Y_{ijk}-\overline{Y}_{...})^{2}\]
F-tester för de olika nollhypoteserna utförs
precis som i one-way layout
.
#
"When such a ratio is substantially
larger than 1
, the presence of an effect is suggested"
Mean square (MS) = Sum of squares / degrees of freedom
Note, for example, that from
#
, \(E(MS_A)=\sigma^2+(JK/(I-1))\sum_i\alpha_i^2\) and that \(E(MS_E)=\sigma^2\).
So if the ratio \(MS_A/MS_E\) is large, it suggests that some of the \(\alpha_i\) are nonzero.
The null distribution of this \(F\) statistic is the \(F\) distribution with \((I-1)\) and \(IJ(K-1)\) degrees of freedom -> can calculate p-value
Randomized Block Design
By comparing fertilizers
within blocks
, the variability between blocks, which would otherwise contribute "noise" to the results, is controlled
Multisample generalization of a matched-pairs design
As a model for the responses in the randomized block design, we will use
\(\alpha_i\) = differential effect of the ith treatment
\(\beta_{j}\) = differential effect of the jth block
\(\varepsilon_{ij}\) = independent random errors
\[Y_{ij}=\mu+\alpha_{i}+\beta_{j}+\varepsilon_{ij}\]
This is the model used earlier, but with the additional
assumption of no interactions between blocks and treatments
. Interest is focused on the \(\alpha_{i}\)
From
#
, if there is no interaction,
\(E(MS_{A})=\sigma^{2}\textrm{+}J/(I-1)\sum_{i=1}^{I}\alpha_{i}^{2}\)
\(E(MS_{B})=\sigma^{2}+I/(J-1)\sum_{j=1}^{J}\beta_{j}^{2}\)
\(E(MS_{\textrm{AB}})=\sigma^{2}\)
Thus, \(\sigma^2\) can be estimated from \(MS_{AB}\). Also, since these mean squares are independently distributed, F tests can be performed to test \(H_A\) or \(H_B\).
For example, to test \(H_{A}:\textrm{ alla }\alpha_{i}=0\), this statistic can be used:\[F=\frac{MS_{A}}{MS_{AB}}\]
From
#
, under \(H_A\), the statistic follows F with \(I-1\) and \((I-1)(J-1)\) degrees of freedom
\(H_B\) may be tested similarly but is
not usually of interest
.
If there
is
and interaction, \(MS_{AB}\) will tend to overestimate \(\sigma^2\) -> F smaller than it should be -> the test is conservative (the actual probability of type I error is smaller than desired
I treatments, J blocks
A Nonparametric method - Friedman's Test
\(H_0\): no treatment effects
Test statistic\[Q=\frac{12J}{I(I+1)}\sum_{i=1}^{I}\left(\overline{R}_{i.}-\frac{I+1}{2}\right)^{2}\overset{a}{\sim}\chi_{I-1}^{2}\]
Within each of the \(J\) blocks, the observations are ranked
\((R_{1j},\ldots,R_{Ij})\)
summan blir ju \(\frac{I(I+1)}{2}\)
så att \(\overline{R}_{..}=\frac{1}{I}(R_{1j}+\ldots+R_{\textrm{Ij}})=\frac{I+1}{2}\)
Useful when \(\varepsilon_{ij}\) are non-normal
Reject \(H_0\) for large \(Q\)
flera samples och flera faktorer samtidigt
Contrary to what this phrase seems to imply, we will be primarily concerned with the comparison of the
means
of the data, not their variances
treatments/levels
where
genomsnitt för viss behandling
genomsnitt över alla behandlingar
\[\uparrow\]
\[SS_{TOT}\]
\[SS_{TOT}=SS_{\textrm{W}}+SS_{\textrm{B}}\]
\[\uparrow\]
\[\uparrow\]
may be expressed symbolically as
\[SS_{W}\]
\[SS_{B}\]
In words, this means that the
total sum of squares
equals the sum of squares
within
groups plus the sum of squares
between
groups.
=ANOVA
:link::page_with_curl:
\[E((X_{i}-\overline{X})^{2})=(\mu_{i}-\overline{\mu})^{2}+\frac{n-1}{n}\sigma^{2}\]
:link::scroll:
\[E(SS_{W})=I(J-1)\sigma^{2}\]
#
insättning
:link::scroll:
\[E(SS_{B})=J\sum_{i=1}^{I}\alpha_{i}^{2}+(I-1)\sigma^{2}\]
#
:link::scroll:
\(SS_{W}/\sigma^{2}\sim\chi^2_{I(J-1)}\)
och om alla \(\alpha_i=0\) så är
\(SS_{B}/\sigma^{2}\sim\chi_{I-1}^{2}\)
och oberoende av \(SS_W\)
:pushpin:\[\dfrac{X_{\gamma_1}^2/\gamma_1}{X_{\gamma_2}^2/\gamma_2} \sim F_{\,\gamma_{1},\,\gamma_{2}}\]om \(X_{\gamma_1}^2\) och \(X_{\gamma_2}^2\) oberoende
:scroll:\[F=\frac{SS_{B}/(I-1)}{SS_{W}/(I(J-1))}\sim F_{\,I-1,\,I(J-1)}\]
#
#
(under \(H_0\))
(under antagandet att
felen
är
normalfördelade
)
#
ptukey(q, nmeans, df) qtukey(p, nmeans, df)
om innehåller noll -> H0 kan rejectas
alla sådana hypotestester kollektivt sett har nivå \(\alpha\)
\[s_{p}^{2}=\frac{SS_{W}}{I(J-1)}\]
Advantage over
#
:
Does not require equal sample sizes
avg. rank in \(i\)th group
avg. rank among all groups
:link::scroll:
\(E(SS_{A})=(I-1)\sigma^{2}\textrm{+}JK\sum_{i=1}^{I}\alpha_{i}^{2}\)
\(E(SS_{B})=(J-1)\sigma^{2}+IK\sum_{j=1}^{J}\beta_{j}^{2}\)
\(E(SS_{\textrm{AB}})=(I-1)(J-1)\sigma^{2}+K\sum_{i=1}^{I}\sum_{\textrm{j=1}}^{J}\delta_{ij}^{2}\)
\(E(SS_{E})=IJ(K-1)\sigma^{2}\)
#
:scroll:
a.
\(SS_E/\sigma^2\sim\chi_{IJ(K-1)}^{2}\)
b.
Under null hypothesis \(H_{A}:\textrm{ alla }\alpha_{i}=0\) så är \(SS_{A}/\sigma^{2}\sim\chi_{I-1}^{2}\)
c.
Under null hypothesis \(H_{B}:\textrm{ alla }\beta_{j}=0\) så är \(SS_{B}/\sigma^{2}\sim\chi_{J-1}^{2}\)
d.
Under null hypothesis \(H_{AB}:\textrm{ Alla }\delta_{ij}=0\) så är \(SS_{\textrm{AB}}/\sigma^{2}\sim\chi_{(I-1)(J-1)}^{2}\)
e.
Ovanstående kvadratsummor är
oberoende
[hur log påverkar CI? additiv -> multiplikativ effekt osv]
\[F=\frac{MS_{A}}{MS_{AB}}\]
