How to choose a statistical test?
Quantitative Data
Categorical Data
Comparing means between groups or a group and a specific value
Independent samples (all involved samples)
Independent samples
Dependent samples
Two Samples
Three or more Samples
Dependent samples (all involved samples)
One Sample
How big is the sample?
Sample size 50 - 300
Sample size > 300
Sample size < 50
Graphical methods:
- Histogram
- box plot
- Q-Q plot
Normality test
Normality test
Really strong normality test
Shapiro-Wilk test (Up to 2000)
Is it normally distributed?
Yes
No / have doubts
Bootstrap methods
How big is the sample?
Sample size 50 - 300
Sample size < 50
Sample size > 300
Normality test for each sample
Normality test for each sample
Normality test for each sample
Really strong normality test
Really strong normality test
Evaluate the behavior in the tails of the distribution
Really strong normality test
Shapiro-Wilk test
Anderson-Darling test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Are all samples normally distributed?
Yes
No / have doubts
Testing homogeneity of variances for each sample
Levene's test
If at least one sample is not normally distributed
Mann-Whitney U test
Are the dispersions homogeneous?
Yes
No / have doubts
Two Sample t-Test
Welch's t-test
Shapiro-Wilk test
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Evaluate the behavior in the tails of the distribution
Assess skewness and kurtosis
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
D'Agostino K^2 test
Anderson-Darling test
Shapiro-Wilk test (Up to 2000)
Are all samples normally distributed?
Yes
No / have doubts
Testing homogeneity of variances for each sample
If at least one sample is not normally distributed
Mann-Whitney U test
Bartlett's test
Are the dispersions homogeneous?
Yes
No / have doubts
Known population variance?
Yes
No
Welch's t-test
Two Sample t-Test
Two Sample z-Tests
Normality test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Is it normally distributed?
No / have doubts
Bootstrap methods
Evaluate the behavior in the tails of the distribution
Really strong normality test
Yes
Shapiro-Wilk test
Anderson-Darling test
One-sample t-Test (Student's t-test)
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Really strong normality test
Shapiro-Wilk test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Known population variance?
No
One-sample t-Test (Student's t-test)
Yes
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Wilcoxon signed-rank test
One-sample z-Test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
How big is the sample?
Sample size < 50
Normality test for each sample
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Are all samples normally distributed?
Yes
Testing homogeneity of variances for each sample
Levene's test
Are the dispersions homogeneous?
Yes
No / have doubts
Welch's ANOVA
No / have doubts
If at least one sample is not normally distributed
Really strong normality test
Shapiro-Wilk test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
one-factor ANOVA
Kruskal-Wallis test
Are there significant differences between the groups?
Yes
No
Tukey's test (Tukey HSD)
Are there significant differences between the groups?
Yes
No
Games-Howell test
Are there significant differences between the groups?
Yes
No
Dunn's method
Sample size 50 - 300
Normality test for each sample
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Really strong normality test
Shapiro-Wilk test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Assess skewness and kurtosis
D'Agostino K^2 test
Sample size > 300
Normality test for each sample
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Really strong normality test
Shapiro-Wilk test (Up to 2000)
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Assess skewness and kurtosis
D'Agostino K^2 test
Are all samples normally distributed?
Yes
Testing homogeneity of variances for each sample
Bartlett's test
Are the dispersions homogeneous?
Yes
one-factor ANOVA
Are there significant differences between the groups?
Yes
Tukey's test (Tukey HSD)
No
No / have doubts
Welch's ANOVA
Are there significant differences between the groups?
Yes
Games-Howell test
No
No / have doubts
If at least one sample is not normally distributed
Kruskal-Wallis test
Are there significant differences between the groups?
Yes
Dunn's method
No
Two Samples
How big is the sample?
Sample size 50 - 300
Normality test for each sample
Really strong normality test
Shapiro-Wilk test
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Sample size < 50
Normality test for each sample
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Are all samples normally distributed?
Yes
No / have doubts
Really strong normality test
Shapiro-Wilk test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Sample size > 300
Normality test for each sample
Really strong normality test
Shapiro-Wilk test (Up to 2000)
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
paired t-test
Wilcoxon signed-rank test
Three or more samples
How big is the sample?
Sample size 50 - 300
Normality test for each sample
Really strong normality test
Shapiro-Wilk test
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Sample size < 50
Normality test for each sample
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Are all samples normally distributed?
Yes
No / have doubts
Really strong normality test
Shapiro-Wilk test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
Sample size > 300
Normality test for each sample
Really strong normality test
Shapiro-Wilk test (Up to 2000)
Evaluate the behavior in the tails of the distribution
Anderson-Darling test
Assess skewness and kurtosis
D'Agostino K^2 test
Not as strong as "Shapiro-Wilk test"
Kolmogorov-Smirnov test with Lilliefors correction
repeated measures ANOVA (RM ANOVA)
Checking sphericity
Mauchly's test
Is sphericity confirmed?
Yes
No / have doubts
Greenhouse-Geisser correction
Friedman test
Are all samples normally distributed?
Yes
No / have doubts
repeated measures ANOVA (RM ANOVA)
Friedman test
Checking sphericity
Mauchly's test
Is sphericity confirmed?
Yes
No / have doubts
the violation of sphericity is not too significant
the violation of sphericity is too significant
Huynh-Feldt correction
Greenhouse-Geisser correction
Wilcoxon signed-rank test