Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter Seven: Scale Reliability and Validity - Coggle Diagram
Chapter Seven:
Scale Reliability and Validity
We must test scales to ensure they measure the unobservable construct that we wanted to measure (validity), and they measure the consistently and precisely (reliability).
Reliability:
degree to which the measure of a construct is consistent or dependable
implies consistency but not accuracy
unreliable observations are often due to an observer's subjectivity
Inter-rater reliability:
aka inter-observer reliability;
measure of consistency between 2 or more independent raters of the same construct
Test-retest reliability:
measure of consistency between two measurements of the same construct administered to the same sample at two different points in time
Split-half reliability:
measure of consistency between two halves of a construct measure
Internal consistency reliability:
measure of consistency between different items of the same construct
"psychometric properties’
of measurement scales
Validity:
aka construct validity;
extent to which a measure represents the what it is supposed to measure
translational validity:
aka representational validity;
how well the idea of a theoretical construct is translated into or represented in an operational measure
face validity:
whether an indicator seems to be a reasonable measure of its underlying construct ‘on its face’
frequency of church attendance, on its face, seems to measure religiosity
content validity:
how well a set of scale items matches with the relevant content domain of the construct that it is trying to measure
typically assessed using a panel of expert judges
criterion-related validity:
how well a given measure relates to one or more external criterion, based on empirical observations
convergent validity:
closeness with which a measure relates to the construct that it's supposed to measure
discriminant validity:
degree to which a measure does not measure other constructs that it is not supposed to measure
often assessed jointly
exploratory factor analysis:
alternative and more common statistical method used to demonstrate convergent and discriminant validity
concurrent validity:
how well one measure relates to other concrete criterion that is presumed to occur simultaneously
do students’ scores in a calculus class correlate well with their scores in algebra?
predictive validity:
degree to which a measure predicts an outcome that it is expected to predict
Does a measure behave the way it should, given the theory of the construct?
integrated approach to measurement validation
must include both theoretical and empirical approaches
integrated approach starts in the theoretical realm
Step One:
Conceptualizing constructs
Step Two:
select or create indicators for each construct
literature review may be helpful
Step Three:
expert judges can examine indicators
prodecure moves into the empirical realm
Step Four:
research instrument is created
1 more item...
Theory of Measurement
classical test theory
or true score theory:
examines how measurement works, what it measures, and what it does not measure
every observation has a
true score
(T) that can be observed accurately if there were no errors in measurement. However, the presence of
measurement errors
(E) results in a deviation of the
observed score
(X) from the true score
X=T+E
measurement errors
random error:
can be attributed to a set of unknown and uncontrollable external factors
considered to be ‘noise’ in measurement and generally ignored
reduces the reliability of measurement
systemic error:
introduced by factors that systematically affect all observations of a construct across an entire sample in a systematic manner
sometimes considered to be ‘bias’ in measurement and should be corrected
reduces the validity of measurement