Survey methods, Scale Development, Psychometrics (Scale Development (Scale…
Survey methods, Scale Development, Psychometrics
- phone online
Scale = instrument for collecting data
- paper-and-pen/ computerised
- established/ newly developed
- open for interpretation
- unrestricted options
- content analysis
- coding systems
- i.e. "think of a day where u felt happy, elaborate etc"
- no ambiguity
- restricted options
- straightforward coding
- with complete descriptions
- with incomplete (anchor) descriptions
Quality of Items
Optimize scale length: reliability increase with # items, but cannot make it too lengthy
item only invites one response
- does not differentiate between respondents
i.e. "I don't think people should avoid doing what they are not told to do" ???? HUH???
in order to choose (dis)agree, have to (dis)agree to 2 parts of the statement
i.e. "I think parking illegally is OK, because everyone does it"
people can infer different things from same statement
i.e. "I dont think Jane should have anything to do with her, because she has no ethics"
- necessary, but not sufficient condition for validity
Consistency of measurement
- "arrows" should hit all around the same area, does not have to be at the bullseye
- observed score = true score + error
- ↑ reliability = smaller error
- ranges from 0 to 1
- if precision is required, need higher reliability
- generally ↑ with # items
Types of Reliability
- stability of measurements over time; require
at least 2 administrations
- Test-retest reliability; how constant are the scores over time 1 and time 2
- construct must remain the same
- Alternate/ Parallel form reliability
if # items is large, divide into 2 versions of scales, correlate the 2 forms
- Split-Half reliability
if # items is small, divide into 2 subsets, correlate the 2 subsets
how items are split may yield different reliability coefficients
- concerns with homogeneity of items in the scale
- Coefficient (Cronbach's) alpha
Accuracy of measurement
- goal is for the "arrows" to hit bullseye
Types of Validity
- most critical to demonstrate this validity!!!
- r/s with other established measures of construct
- measures of SAME construct
- scale correlate with measures of other constructs that it is theoretically supposed to correlate with
- if no access to existing measure of same construct:
Known-groups validation i.e. use same measure, on 2 groups of people where a difference in scores is expected
- measures of DIFFERENT construct
- scale does not correlate with measures of other constructs that it is theoretically not supposed to correlate with
- do items in the scale look like they assess the construct of interest??
- do items in the scale cover different facets of the construct??
i.e. adequate sampling of content domain
- r/s with a criterion; does scale relate to an independent criterion?
- only need to test this validity if the scale is specified to be used with a criterion
- if criterion is assessed at same time
- if criterion is assessed at a later time
issues related to sampling:
- random sampling = to generalise to population
- convenience sampling = most of the time, surveys are convenient samples