EXPERIMENTS

Causality

Experimental Designs

Validity

Lab vs Field Experiments

Internal Validity: extent to which the observed results are due to the experimental manipulation
--> low internal validity --> low external validity

External Validity: extent to which the observed results are likely to hold beyond the experimental setting
--> whether results will replicate in other areas, if don't have internal validity, results won't replicate in other context.
--> must have high internal to have high external

Ecological Validity: methods, materials and setting of the study approximate the real-world
--> isolate effects v carefully, stimuli not rich enough to capture real-world, keep everything constant

Experimental Notation

Common Designs

  • Causality: change in one variable (X, IV) will produce a change in another variable (Y, DV)
    --> e.g. an increase in temperature (X) increases click-through (Y) for Molson's Coors social media ads
  • Establishing causality is the main reason why we conduct experiments
    --> IV responsible for change in DV
    --> IV changes, DV measured against

Correlation DOES NOT EQUATE to Causation

  • X and Y may be correlated bc
    --> X change Y
    --> Y change X
    --> Z change X and Z change Y
    -Z is a confound i.e. an extraneous variable that influences both X and Y
    -Confound must change BOTH X and Y

3 Conditions for Causality

  1. X must occur before Y
  2. There must be an association b/w X and Y i.e. a change in X must lead to a change in Y
    --> must be correlated
  3. There must be no other competing explanations for the r/s
    --> cannot have alternative explanations for observed r/s

O - Any formal observation/ measurement
X - Exposure to the treatment
EG - Experimental grp
CG - Control grp
R - Random assignment
M - Matching control

Classical Designs

  • Pre-experimental Designs:


    --> one grp, after only


    --> one grp, before after


    --> non-matched control


    --> matched control


  • Quasi-experimental Designs:


    --> time-series


    --> continuous panel


  • True Experimental Designs: (w randomisation)


    --> two grp, after only


    --> two grp, before after


    --> solomon four grp

Statistical Designs

  • Completely randomised
  • Randomised block
  • Factorial
    --> to vary multiple variables simultaneously
    --> allows testing of main and interaction effect (no/ +ve/ -ve synergy)

only one treatment level

more than one treatment level

Threats to Internal Validity

  • History: external events that affect the responses of the ppl involved in the experiment e.g. competitor gg out of biz
  • Maturation: changes in the respondents that are a consequence of time e.g. aging
  • Testing: the fact that a participant has alr been previously measured may affect their future behaviour
  • Instrumentation: changing measurement instrument
  • Selection bias: EG (not sample) is systematically diff from population
  • Mortality: respondents consistently drop out of either EG or CG while experimental research is in progress

Threats to External Validity

  • All threats to internal validity
  • Hawthorne effect: respondents behave differently bc they know they are being observed

Lab Experiment

  • Randomisation is easy and external factors can be controlled
    --> high internal validity
  • Subjects are in an artificial environment and aware they are being observed
    --> low ecological validity

Field Experiment

  • Subjects are in natural environment
    --> high ecological validity
  • Subjects are unaware of observation, but randomisation may be hard (e.g. price discounts in supermarkets) or easy (e.g. A/B testing)
    --> high external validity?