ME WHEN THE STATIS STICKS (part 2)
Statistical Analyses
ANOVA
F value
Ranges 1 - infinite
Effect Size Indicator
Eta Squared (η^2)
f
Omega Squared (ω^2)
used when N < 30
needed to calculate G*Power
Correction Procedures
Family-Wise Type I Error Rate (3 levels of IV) .143 (14.3%)
Fisher's Protected Least Signficant Difference (LSD)
Bonferroni Correction
Tukey-Kramer's Honestly Signficant Difference (HSD)
Research Methodology
Ratio of ''good stuff' to ''bad stuff''
Types of Research Questions
Frequency
Association
Causal
Three Conditions
Relationship Condition
Temporal Order Condition
No Alternative Explanation
X and Y are associated
Changes in X precede changes in Y
No plausible alternate explanation exist
Research Approaches
Experiment
Quasi-Experiment
Control Techniques
Natural Manipulation
Longitudinal Correlation
Cross-sectional Correlation
Objective Observation
Of Phenomena That Are Made to Occur
In a Strictly Controlled Situation in which One or More Factors are Varied, While All Others are Kept Constant
Control Group
Random Assignment
RNGs
Disadvantages
Not all variables can be manipulated
Some phenomena are unethical to experiment with
Experiments can be costly
Experiments have their own weaknesses
Types of Validity
Construct
Statistical Conclusion
External Validity
Internal Validity
Threat to Internal Validity
Testing
History
Instrumentation
Regression Towards the Mean
Maturation
(Experimental) Attrition
Selection
Selection Interactions
Other Threats
Participant Effects
Experimenter Effects
Sequencing Effects
Experimental Designs
Don't use pretest measure
Use Alternate Form For Pre- and Posttest
Wash-out Period
Calibrate Instruments Regularly
Train Human Raters
Give Them Breaks
Document Reasons For Leaving the Study
Random Attrition: OK
Differential Attrition: Bad
Reactivity
Demand Characteristics
''playing good''
''playing bad''
Expectancy Effects
Attribute Effects
Omit Information or Deceive Participants
Blinding
Automation
Same Experimenter Collects Data
Experimenter Leaves Room When Data Collected
Order Effects
Carryover Effects
Counterbalancing
Washout Period
Weak Designs
Quasi-Experimental Designs
Strong Experimental Design
One-Group Posttest Only
One-Group Pretest-Posttest
Posttest-Only Non-Equivalent Groups
Non-Equivalent Groups Comparison Group
Time Series
Posttest-Only Control Group
Within-Participants Posttest-Only
Pretest-Posttest Control Group
Matching
Holding Variables Constant
Building EVs into Design
IV
CV
Yoked Control
Equating Participants
NHST
Null Hypothesis
Alternative Hypothesis
Errors
Type I Error
Type II Error
False Positive
False Negative
Power
1 - β
β = Probability of Type II error
1 - β = .80
β = .20
Power Analysis
A Priori
Post Hoc
Sensitivity
Valid Positive
N = ?
1 - β = ?
Effect Size = ?
Effect Size
Degress to Which H0 is False
d
r
f
eta squared & partial eta squared
Small = .20
Medium = .50
Large = .80
Small = .10
Medium = .30
Large = .50
Small = .10
Medium = .25
Large = .40
Small = .01
Medium = .06
Large = .14
Procedure For Conducting Experiment
Ethical Approval
Preregistering Hypothesis
Stop HARKing
Sampling Approaches
Random
Non-random
Sample Size (N)
Convenience
Purposive
Snowball
Experimental Setting
Apparatus/Instruments and Procedure
Consider Automation
Laboratory
Field
Online
click to edit
Piloting
Ensures Strong Manipulation of IV
Consent
Debrief
Dissemination of Findings
IV little effect, F value close to 1
IV greater effect, greater F value
Multiple Comparisons
General Term for any kind of test which is more specific than the omnibus F test
Pairwise Comparison
Non-Pairwise Comparison
Post-Hoc Testing
Planned Comparisons (Planned Contrasts
Comparison of 2 groups only
Comparison in which study groups are being combined
All possible pairwise comparisons in a study are being made
Only select comparisons are being made (specified before starting the study).
Check Omnibus F test is sig
Conduct Multiple Comparisons without adjustment to p value
Divide α by the number of comparisons to be made and then use that as your new α level
Remember that SPSS multiplies p value
check table for *Bonferroni correction applied (i.e do not apply correction twice accidentally)
SPSS makes p value adjustment
Can be conservative
Keeps FW Error Rate at .025 rather than .05
Appropriate when ns being compared are of similar sizes and have similar variances
Factorial Designs
One-way Designs
1 IV (more than 2 levels)
Factorial Designs
More than 1 IV
2 IVs
Main Effect IV 1
Main Effect IV 2
Interaction Effect ( IV 1 * IV 2)
Interaction
When the effect of an IV on the DV depends on some other variable
Interpret Main Effects with Caution
Simple Effects Analysis
Types of ANOVA
How Many IVs?
One
Two(+)
IV is Between-Subjects Variable
IV is Within-Subjects Variable
One-way between-subjects ANOVA
One-way within-subjects ANOVA
All between-subjects variables
All within-subjects variables
Mix of between- and within-subjects variables
Factorial betwen-subjects ANOVA
Factorial within-subjects ANOVA
Mixed-model ANOVA
Variance in DV
Once individual differences subtracted, error term smaller than in corresponding between-subjects design
Increased Sensitivity
All Assume (Assumptions)
Random Sampling
Independence
DV is Scale
Normality
BW-S: No participant participates more than once
WI-S: Same participants are measured in all measures
Non-Parametric Tests
Kruskal-Wallis ANOVA
Friedman ANOVA
If Omnibus F test sig, follow up wuth series of Mann-Whitney U tests (Bonferroni Correction)
If Omnibus F test sig, follow up wuth series of Wilcoxon Signed Rank tests (Bonferroni Correction
Compare Mean Ranks as opposed to Means
Other Tests
ANCOVA
MANOVA
Combines elements of ANOVA and Regression to compare group menas, after adjustment for one or more CVs
Reduces Confounding
Useful for quasi-experiments with non-equivalent groups
Increases power
Effect of CV on DV is removed
More than 1 DVs
Compares groups in terms of a linear composite DV
Controls FWT1ER
Establish sig. omnibus F test
Then run series of ANOVAs (one for each DV)
Work best when DVs are moderately correlated, r = .60
Other suggest it is simpler to perform Bonferroni Correction
No more than .90 (violates multicollinearity)
Pairwise comparisons can be performed on DV
but must be done via syntax in SPSS
Not optimal, but most common approach
Confidence Intervals
Rules to determine group differences based on the degree of CI overlap
If CIs slight gap, means differ at < .01
If CIs slight overlap, means differ at p = .01
If CIs moderate overlap, p = .05~
Rules only apply to BW-S data
Assumptions
Assumptions
Shape of Distribution of DV
DV at least Ordinal
Symmetry of the distribution of differential scores
DV at least ordinal
Homogeneity of Variance
BW-S
WI-S
Levene's Test
F-Max Test
Assumption of Sphericity
Mauchly's Test
If more than 2 WI-S Variables
Assumption of Homogeneity of Covariance Matrices
Box's Test
Assumption of Homoegenity of Regression Slopes
Assumptions
Univariate and Multivariate Normality
Cell Size
Absence of Mulitcollinearity