Lecture 11 - Item Response Theory

Facts about IRT

  • Items only have two variables (Y/N)
  • Used for: personality + ability
  • Based on common factor model:
  1. Items are “reflective” ⇒ they are caused by the latent variable
  1. Items are unidimensional (one latent variable)

In IRT we assume:

  1. The distribution of latent variables are interval and normally distributed (M = 0, SD = 1)
  1. We assume unidimensional model (only one factor is influencing out items)
  • What is IRT? It is common factor model + logistic regression
  • IRT = independent/predictor - predicts item 1
  • Theta is always the latent variable

Item Characteristic Curve (ICC) = Another term for logistic regression curve in the context of IRT

Applications of IRT: (1) Creating tests for specific target audiences, for example measuring IQ of "gifted individuals" (aka test linkage) (2) Development of (computer) adaptive test (every person their own test, level of items is adapted to your level based on your given answer (3) Testing construct bias (4) Investigate likelihood of response pattern (for example, to investigate response bias)

What is IRT?:
IRT is a single common factor model with dichotomous item response format with ordinal items:

  • (Y/N), (1/0)
  • Since there is only 2 categories, then there is no linear regression, but there is logistic regression
  1. Item scores are causally and directly dependent on the latent variable

Theory about IRT: The latent variable (Independent, theta) predicts the probability of item response = 1 (dependent) - we use probability because IRT = logistic regression

The One Parameter Logistic(1PL) model:

It is called ONEPL model because only the b-parameter differs across items

  • b = difficulty or location parameter
  • We will get many b parameters, but only one a parameter
  • The value of theta is called parameter b
  • The b parameter is called the location parameter because it tells us where the logistic regression line(ICC) is located along the theta dimensions (x-axis)

What determines the probability of answering an item correct? (1) Difficulty (b) and (2) level of ability of a person

Bell curve = the higher the theta on the x-axis the more difficult the questions

About the b parameter: When the b parameter is changed: the curve will shift to the left or right. If the item is easier then the line will shift to the left + the b parameter tells us where on the theta dimension the item discriminates most, and how well it discriminates. Easier items discriminate at a lower level

IRT is mostly used for cognitive traits, but it can also be sued for measuring fx OCD (=higher level of OCD => the further to the right the line will be)

The Two Parameter Logistic (2PL) model:

How is the 2PL model different from the 1PL model?

b tells me where the line is steepest, a parameter tells us how steep the line is

  • If the a parameter = 0, then the items does not discriminate at all => answering the item correct is 50/50
  • What is equivalent of a? = factor loadings from the common factor model
  • Remember that a differs across items. So 5 items will have 10 unique parameters (5 b parameters, and 5 a parameters)

You can think of discrimination as: Does a slight increase/decrease in theta lead to any noticeable differences in the probabilities of answering yes?

Questions about the b parameter:

Q: How do we interpret b?
A: Gives where on the theta dimension the probability of "item correct" or "agree" is 0.5


Q: How does b influence the item characteristic curve (ICC)?
A: Determines where in the graph the curve will be (at which location, low = left, high = right)


Q: Why is b important?
A: b tells us where on the theta dimension the item discriminates most

We know that discrimination of an item is maximal at theta = b. This means that conditional on theta = b, the variance of the item responses is maximal.
If the slope a=0, then the item does not discriminate at all

Questions about the a parameter:

Q: How do we interpret a?
A: The a-parameter tells us how well the discrimination/steepness of the slope is at Theta = b


Q: How does a influence the item characteristic curve (ICC)
A: the larger a, the steeper the curve


Q: Why is a important?
A: Shows how good the item is: the larger the a-parameter, the stronger the item is an indicator of the latent trait (think about factor loadings/communalities)

Most important differences between 1PL and 2PL


  1. 1PL: Equal discrimination parameters:
    a=1, b=-2,-1,0,1,2

2PL: Unequal discrimination parameters
a=0.62,1.20,0.94 etc
b=-2,-1,0,1,2


2.
For the 1PL model the ICCs are parallel (do not cross)

Most important similarity: b-parameter equals value of theta where probability of score 1 given theta equals 0.5