Please enable JavaScript.

Coggle requires JavaScript to display documents.

1. RELIABILITY (INTERPRETE IT! (:!: Special Circumstances (Measuring…

- - - - Unsystematic:
        Item selection / Test administration / Test scoring (subjectivity)
      - Systematic: Consistently wrong (validity problem)
    - - Unsystematic MEs = random / Mean Me = 0 / True scores and unsystematic MEs = uncorrelated / True score = constant / Standard distribution of MEsame for each participant / MEs not correlated with MEs from other tests / Minimal or absent systematic MEs / similar abilities
  - - - Each Item has own IRF/ICC! (Each trait level has own ME)
        
        Difficulty level =
        how much of trait needed to answer item correctly on specific trait level θ
        
        Item discrimination paramenter =
        how well item differentiates of specific trait level (steeper slope)
      - X-axis: Trait level (theta)
        Y-axis: Probability of correct answer
      - True score ( θ ) = higher θ , higher trait
    - - Undimensionality / Local independence / Timeless / Discrimination parameters same for all items (in Rasch Model)
    - - Scale Information Functions = add info from diff. scale levels together (shows reliability of test on diff θ -levels)
    - - 1-Parameter (Rasch) = probability of respondent with θ correctly responding to item with difficulty b
      - 2-Parameter = adds item discrimination index
      - 3-Parameter = adds guessing
- - - - :red_cross: ME-variance-source: changes over time (carryover effect, practice effect) / overestiamtion of realiability possible / systemtic errors have no infuence on score variability / motivation fluctuations
      - :check: realiable if: high r between two tests (can predict 2nd score from1st score) / measuring constant variables (abilities /personlity)
    - - Immediate
        
        :red_cross: ME-variance-source: item sampling / expensive / motivation fluctuations / when tests not parallel = underestimation of realiability
      - Delayed
        
        :red_cross: ME-variance-source: same as immediate + changes over times
  - - - Odd-even System (only if items odered acc. to difficulty level)
      - :check: reliable if: 2 halves show strong correlation r divided by 2 (two administrations should also show high r) / or Spearman-Brown-Formula (only if two halves have same SD) / takes increasing difficulty-level into account
      - :red_cross: ME-variance-source: internal consistency / item sampling / nature of split
    - - Index of interrelatedness of individual items, internal consistency, low uniqueness (NOT unidimesionality = you can measure 2 diff. factors but still have high alpha )
      - Assumptions: unidimensionality / factorial purity / for continous items (e.g. scale)
      - :warning: large scales have high alpha (many items) / decision-dependent adequate realiability (context) / not a valiidity-measure (only says how correlated halves are) / only useful for factorial simple tests / lower bound of test-reliability (only equals realibility if tau-equivalent) / says nothing about error-s
      - :red_cross: ME-variance-source: item sampling / test heterogenity (e.g. IQ)
      - :check: good estimate of split-half reliability / less susceptible to randomness / not dependent on split / more general than KR20
      - Alpha-"Precision" = SE of inter-itemcorrelation (large SE: symptom of multidimensionality, not proof) / also decreases if more items in test
        
        :!: E.g. Alpha=0.8, Mean inter-itemcorr.=0,57 (3-item scale) OR 0.28 (10-item scale)
      - Types
        
        Standardized Item Alpha (average inter-itemcorrelation, stepped up with Spearman-Brown-F.) - bigger than Cronbach's Alpha
        
        :check: if item standard scores summed to form scale-scores / :red_cross: if you calculate with simple raw scores & does not take diff. SD into account
      - Flanagan-&-Rulon-Formula = takes diff. SD into account (unlike Spearman)
    - - :red_cross: Need to know proportion of people who answered correctly, but many items do not have correct or wrong options
    - - :warning: supplement
      - :red_cross: ME-variance-source: scorer diff. (subjectivity)
      - Kappa: <0.6 (bad) / 0.6-0.8 (sufficient) / >0.8 (good)