Reliability: Theoretical Basis (Wk 3)

Classical Test Theory (CTT)

Four Conceptions of Reliability

Standard Error of Measurement

Types of Scores in CTT: Observed score / True score / Error score variability

Parallel Test

Definition of Reliability

Not an all-or-nothing thing (Reliable / Not Reliable) ❎

Consistency of Measurement

Exists on a Continuum (More Reliable / Less Reliable )✅

CTT in General

Function

Assumption

Definition

A measurement theory that defines reliability (using theoretical basis)

Reliability is the degree in differences with observed scores that are consistent with differences in their true scores.

Outlines procedures for estimating reliability of psychological measures

Types of Scores in CTT

Measurement Error (ME)

Observed Score

True Score

True score + measurement error = Observed Score

Creates inconsistency between observed + true scores

Actual amount of psychological characteristics being measured that the respondent possesses

Source of ME

Measurement Error cannot be fully omitted, so scores can be unreliable to some degree.

Component of observed score > nothing to do with psychological characteristic that is being measured

Test Construction

Test Administration

Test Scoring and Interpretation

The only amount / score we know when testing (as it's contaminated by ME)

ME is random: likely to inflate / deflate the score

Error cancel itself out across respondants

Error scores are uncorrelated with true scores

Variance in Scores

Depends on the link between observed, true & error score

FORMULA: Relationship of the 3 variances (squared)

The need for consistency between (true & observed score) differences

Four Conceptions of Reliability

click to edit

Terminology