ME WHEN THE STATIS STICKS (part 2)

Statistical Analyses

ANOVA

F value

Ranges 1 - infinite

Effect Size Indicator

Eta Squared (η^2)

image

f

Omega Squared (ω^2)

image

used when N < 30

needed to calculate G*Power

Correction Procedures

Family-Wise Type I Error Rate (3 levels of IV) .143 (14.3%)

image

Fisher's Protected Least Signficant Difference (LSD)

Bonferroni Correction

Tukey-Kramer's Honestly Signficant Difference (HSD)

Research Methodology

Ratio of ''good stuff' to ''bad stuff''

Types of Research Questions

Frequency

Association

Causal

Three Conditions

Relationship Condition

Temporal Order Condition

No Alternative Explanation

X and Y are associated

Changes in X precede changes in Y

No plausible alternate explanation exist

Research Approaches

Experiment

Quasi-Experiment

Control Techniques

Natural Manipulation

Longitudinal Correlation

Cross-sectional Correlation

Objective Observation

Of Phenomena That Are Made to Occur

In a Strictly Controlled Situation in which One or More Factors are Varied, While All Others are Kept Constant

Control Group

Random Assignment

RNGs

Disadvantages

Not all variables can be manipulated

Some phenomena are unethical to experiment with

Experiments can be costly

Experiments have their own weaknesses

Types of Validity

Construct

Statistical Conclusion

External Validity

Internal Validity

Threat to Internal Validity

Testing

History

Instrumentation

Regression Towards the Mean

Maturation

(Experimental) Attrition

Selection

Selection Interactions

Other Threats

Participant Effects

Experimenter Effects

Sequencing Effects

Experimental Designs

Don't use pretest measure

Use Alternate Form For Pre- and Posttest

Wash-out Period

Calibrate Instruments Regularly

Train Human Raters

Give Them Breaks

Document Reasons For Leaving the Study

Random Attrition: OK

Differential Attrition: Bad

Reactivity

Demand Characteristics

''playing good''

''playing bad''

Expectancy Effects

Attribute Effects

Omit Information or Deceive Participants

Blinding

Automation

Same Experimenter Collects Data

Experimenter Leaves Room When Data Collected

Order Effects

Carryover Effects

Counterbalancing

Washout Period

Weak Designs

Quasi-Experimental Designs

Strong Experimental Design

One-Group Posttest Only

One-Group Pretest-Posttest

Posttest-Only Non-Equivalent Groups

Non-Equivalent Groups Comparison Group

Time Series

Posttest-Only Control Group

Within-Participants Posttest-Only

Pretest-Posttest Control Group

Matching

Holding Variables Constant

Building EVs into Design

IV

CV

Yoked Control

Equating Participants

NHST

Null Hypothesis

Alternative Hypothesis

Errors

Type I Error

Type II Error

False Positive

False Negative

Power

1 - β

β = Probability of Type II error

1 - β = .80

β = .20

Power Analysis

A Priori

Post Hoc

Sensitivity

Valid Positive

N = ?

1 - β = ?

Effect Size = ?

Effect Size

Degress to Which H0 is False

image

d

r

f

eta squared & partial eta squared

Small = .20

Medium = .50

Large = .80

Small = .10

Medium = .30

Large = .50

Small = .10

Medium = .25

Large = .40

Small = .01

Medium = .06

Large = .14

Procedure For Conducting Experiment

Ethical Approval

Preregistering Hypothesis

Stop HARKing

Sampling Approaches

Random

Non-random

Sample Size (N)

Convenience

Purposive

Snowball

Experimental Setting

Apparatus/Instruments and Procedure

Consider Automation

Laboratory

Field

Online

click to edit

Piloting

Ensures Strong Manipulation of IV

Consent

Debrief

Dissemination of Findings

image

IV little effect, F value close to 1

IV greater effect, greater F value

Multiple Comparisons

General Term for any kind of test which is more specific than the omnibus F test

Pairwise Comparison

Non-Pairwise Comparison

Post-Hoc Testing

Planned Comparisons (Planned Contrasts

Comparison of 2 groups only

Comparison in which study groups are being combined

All possible pairwise comparisons in a study are being made

Only select comparisons are being made (specified before starting the study).

Check Omnibus F test is sig

Conduct Multiple Comparisons without adjustment to p value

Divide α by the number of comparisons to be made and then use that as your new α level

Remember that SPSS multiplies p value

check table for *Bonferroni correction applied (i.e do not apply correction twice accidentally)

SPSS makes p value adjustment

Can be conservative

Keeps FW Error Rate at .025 rather than .05

Appropriate when ns being compared are of similar sizes and have similar variances

Factorial Designs

One-way Designs

1 IV (more than 2 levels)

Factorial Designs

More than 1 IV

2 IVs

Main Effect IV 1

Main Effect IV 2

Interaction Effect ( IV 1 * IV 2)

Interaction

When the effect of an IV on the DV depends on some other variable

Interpret Main Effects with Caution

Simple Effects Analysis

Types of ANOVA

How Many IVs?

One

Two(+)

IV is Between-Subjects Variable

IV is Within-Subjects Variable

One-way between-subjects ANOVA

One-way within-subjects ANOVA

All between-subjects variables

All within-subjects variables

Mix of between- and within-subjects variables

Factorial betwen-subjects ANOVA

Factorial within-subjects ANOVA

Mixed-model ANOVA

Variance in DV

Once individual differences subtracted, error term smaller than in corresponding between-subjects design

Increased Sensitivity

All Assume (Assumptions)

Random Sampling

Independence

DV is Scale

Normality

BW-S: No participant participates more than once

WI-S: Same participants are measured in all measures

Non-Parametric Tests

Kruskal-Wallis ANOVA

Friedman ANOVA

If Omnibus F test sig, follow up wuth series of Mann-Whitney U tests (Bonferroni Correction)

If Omnibus F test sig, follow up wuth series of Wilcoxon Signed Rank tests (Bonferroni Correction

Compare Mean Ranks as opposed to Means

Other Tests

ANCOVA

MANOVA

Combines elements of ANOVA and Regression to compare group menas, after adjustment for one or more CVs

Reduces Confounding

Useful for quasi-experiments with non-equivalent groups

Increases power

Effect of CV on DV is removed

More than 1 DVs

Compares groups in terms of a linear composite DV

Controls FWT1ER

Establish sig. omnibus F test

Then run series of ANOVAs (one for each DV)

Work best when DVs are moderately correlated, r = .60

Other suggest it is simpler to perform Bonferroni Correction

No more than .90 (violates multicollinearity)

Pairwise comparisons can be performed on DV

but must be done via syntax in SPSS

Not optimal, but most common approach

Confidence Intervals

Rules to determine group differences based on the degree of CI overlap

If CIs slight gap, means differ at < .01

If CIs slight overlap, means differ at p = .01

If CIs moderate overlap, p = .05~

Rules only apply to BW-S data

Assumptions

Assumptions

Shape of Distribution of DV

DV at least Ordinal

Symmetry of the distribution of differential scores

DV at least ordinal

Homogeneity of Variance

BW-S

WI-S

Levene's Test

F-Max Test

Assumption of Sphericity

Mauchly's Test

If more than 2 WI-S Variables

Assumption of Homogeneity of Covariance Matrices

Box's Test

Assumption of Homoegenity of Regression Slopes

Assumptions

Univariate and Multivariate Normality

Cell Size

Absence of Mulitcollinearity