Please enable JavaScript.
Coggle requires JavaScript to display documents.
Psychometrics Modules 9: Construct Validity (a contemporary conception of…
Psychometrics Modules 9: Construct Validity
introduction
construct validity may be a more comprehensive approach, like an umbrella, that includes all types of reliability and validity
recognizes that the concepts are abstract
example would be big five, it is subject to observation and the scale we use. it is not concrete like weight or height
a way of developing a good measure or construct is comparing it to a similar construct and measure, we can compare our constructs and measure to that construct and measure to get the most reliable and valid measures possible
if scores are similar from our construct to the construct we chose to compare it to then we have
convergent validity
if scores from our construct and the construct we have chose to compare it to do not match up then we have
discriminate validity
multitrait-multimethod matrices
CMV
: refers to a problem in which correlations between constructs are artificially inflated because the data were obtained using he same method of data collection for each variable
CMV
results in a correlation not because there is an underlying relationship between the two measures but rather because we used the same measures to obtain data for the two different constructs
today we now use the
MTMM
matrix because we can systematically assess the relationships between two or more constructs (i.e. traits), each of which is measured using two or more methods. data is collected from a single sample of individuals. the MTMM matrix is the resulting correlation matrix between all pairs and measures.
using
MTMM
when a trait is similar to another trait and there is a high correlation it is valid regardless of the method used to obtain either trait. divergent validity is evident when there is a low correlation rate between two traits.
a
MTMM
matrix is capable of examine the degree to which
CMV
influences the observed correlations between two variables.
to produce evidence of construct validity in this way, the pattern of correlations within the MTMM matrix must provide evidence that our measure of a trait of interest correlates higher with theoretically similar constructs that are measured using different methods than our measure of interest correlates with measures of theoretically dissimilar constructs, whether measured using the same measure or not.
additional aspects of construct validation
studies of group differences
: if two groups are expected to differ on a construct, do they indeed differ as expected?
studies of internal structure
: if a test is put forth as measuring a particular construct, then the items on the test should generally be interrelated. this means that alpha can provide evidence of construct validation.
studies of stability of test scores
: we would expect measures of enduring traits to remain stable over time, whereas measures of other constructs are expected to change over time, such as following an intervention or experimental treatment. construct validation evidence can be garnered based on whether test scores reflect the expected stability over time.
studies of process
: unfortunately, differences in test scores are sometimes determined by more than just the construct the researcher intended to assess. verbalizing the items can help us understand if the participant understands the items.
a contemporary conception of construct validation
construct underrepresentation
: refers to measurement that fails to capture the full dimensionality of the intended construct. occurs when important aspects of a construct are not measured in a test.
construct irrelevant variance
: refers to measurement of reliable variance that is not part of the construct of interest. something is measured by the test that is not relevant to the construct.
content
: the boundaries to which the construct domain is to assess. concerned with both relevance and representativeness of the measure.
substantive
: concerned with the content aspect by suggesting the need to include "empirical evidence of response consistencies or performance regularities reflective of domain processes" the relevant measures to the construct of the domain are empirically tested.
structural
: ensuring that the construct domain determines the rational development of construct-relevant scoring criteria and scoring rubrics. emphasize the importance of score comparability across different tasks and different setting.
generalizability
: asserts that the meaning of the test scores should not be limited merely to the sample of tasks that constitute the test, rather should be generalizable to the construct domain intended to be assessed. evidence regarding the generalizability of test scores would help determine the boundaries of the meaning of the test scores.
external
: refers to the empirical relationships between the test scores and scores on other measures. examines whether the empirical relationships between test scores and other measures is consistent with our expectations. includes aspects of the elements of convergent and discriminant validity, as well as criterion-related validity
consequential
: the examination of evidence regarding the consequences of score interpretation. in other words, what are the intended as well as unintended consequences of testing? recommends that conclusions be drawn from long periods of testing or longitudinal testing. the primary concern is to ensure that any negative consequences of test usage are unrelated to sources of test invalidity