Please enable JavaScript.
Coggle requires JavaScript to display documents.
Module 7: Content Validity (the role of expert judgement (once the test is…
Module 7: Content Validity
a popular definition of validity is whether a test measures what it is intended to measure
more accurately, the process of validation does not seek to determine whether the test itself is valid, but rather whether the inferences and conclusions that are made on the basis of test scores are valid
the role of expert judgement
once the test is developed, SMEs can determine if the test will measure what it is intended to measure
SMEs should be composed of competent supervisors for example, they should also be diverse, gender inclusive, and geographically diverse as well
in module 4 we discussed defining the domain, in module 7 we can see how that comes up again
content validation is most appropriate for domains that are concrete and clear, not abstract
content validity: a simple example
if a history teacher makes an exam on chapters 1-6 but 75% of the questions come from chapter 5 then the test does not measure all of the domains equally
formalizing content validity with the content validity ratio
in assessing CVR, the panel of SMEs is asked to examine each item on the test. for each item they rank it as either "essential" "useful" or "not necessary"
acceptable size of CVR
can range from +1 to -1 on any item. the closer to +1 the more essential it is, 0 would be useful, -1 would be not necessary. the more SMEs there are that rate the test the lower the alpha can be. 0.5 is still relevant like most other cases with alpha
the content validity index
measures the validity of the test as a whole
for the example with the history teacher, it is likely the test created would have a high CVI but it does not go over the whole content domain
a second approach to formalizing content validity
it is called the content validation form or CVF
measures the test as a whole
does an item-by-item analysis
measures symptoms of questionable tests
face validity
this is when test takers themselves try to determine if the test they are taking is valid
can influence test takers
performance
and
motivation
concluding comments
have to make sure it measures the entire content domain
selection of appropriate SMEs is important to the validity of the test
