Please enable JavaScript.

Coggle requires JavaScript to display documents.

Validity (Chapter 9 (evaluating discriminant & convergent validity…

- - - - examining a few very relevant variables
      - if test scores are indeed correlated to the relevant variables
        
        validity coefficients high
        
        affirming evidence for the use of test
      - validity generalization
        
        small studies' results are difficult to generalize
        to other situations
        
        evaluating validity coefficients across many small-sample studies
        
        can reveal general predictive validity across individual studies
        
        can reveal degree of variability across individual studies
        
        can reveal the source of variability across the studies
    - - examining many correlations
        
        between many criterion variables and main construct
      - assessing whether correlations 'make sense' according to conceptual basis of construct
        
        somewhat subjective
    - - measures of several traits, each which is measured by multiple methods
      - forming guidelines for evaluating discriminant & convergent validity
      - examines two sources of variance --> method & trait variance
      - correlations examined through psychological and methodological ways
        
        i.e. shared common variance
        
        strong correlation suggests measures trait & method variance
        
        weak correlation suggests measures don't share method & trait variance
    - - how well theoretical predictor correlations and actual predictor correlations fit
      - QCV in 3 phases
        
        1) predictions about pattern of discriminant & convergent validity correlations
        
        2) data collection + computation of actual correlations
        
        3) quantifying the extent to which predictor & actual correlations fit
      - results: 2 effect sizes + significance test
  - - - variance explained
        
        used in regression analyses, ANOVA
      - squared correlation
        
        "proportion of variance in one variable that is explained by the other"
      - criticism:
        
        sometimes correlation itself could be used as 'variance explained'
        
        variance is on a non-intuitive metric
        
        squaring correlation makes association seem smaller than in reality
    - - binomial effect size display
        
        100 are high-scorers, 100 are low-scorers
        
        limitations
        
        assumes equal probabilities
        
        making predictions for 200 people group
      - Taylor-Russell table
        
        useful when proportions are not equal
        
        dichotomous variables
        
        popular in hiring decisions
      - utility analysis
        
        validity in terms of costs-gains relationship
        
        monetary values assigned to aspects of tests
      - test sensitivity & specificity
        
        sensitivity
        
        probability that a test correctly identifies those with the disorder
        
        specificity
        
        probability that a test correctly identifies those who don't have the disorder
        
        4 possible outcomes
        
        true negative, true positive, false negative, false positive
- - - - interpretation can be valid/invalid
    - - spectrum, rather than an absolute value
    - - operationalism
    - - low correlation
        
        low validity
    - - not necessarily high validity
    - - a wrong interpretation of validity can be dangerous
        
        misleading, leads to wrong treament, etc.
  - - - match between item content, and content that should be included
        
        content needs to be representative of measured psychological attribute
        
        dangers: construct-irrelevant content, construct underepresented
    - - as evaluated by non-experts (i.e. participants)
      - how much the measure seems to be related to psychological construct
      - may influence participants' motivation, honesty, biases, etc.
    - - match between actual test content + content test should have (according to theory)
      - way parts of test are related to each other
      - factor analysis is used to investigate this
    - - match between psychological attributes respondents use vs. which they should use
      - responses may/may not reflect the intended psychological process
    - - match between measurement's actual associations with other variables vs. which associations it should have
      - convergent + discriminant validity
      - concurrent validity
        
        degree to which test scores are related with relevant variables measured at the same time
    - - degree to which test scores have same consequences for different societal groups
      - combines issues of politics with science
    - - how test scores relate to a relevant variable(s) measured at a future point
    - - whether a new psychometric test would increase predictive validity
    - - degree to which test scores are related
    - - a test can have negative consequences for a group, but still be fair
      - a test can be biased so a certain group performs worse
    - - how well does the test differentiate between groups of people
      - how well the test makes predictions
      - association between test score and criterion variable
      - concurrent + predictive validity
      - psychological meaning of test may be irrelevant
    - - emphasis on meaning of test score
      - not concerned so much with specific hypotheses
      - unbiased level of specific psychological attribute