Please enable JavaScript.

Coggle requires JavaScript to display documents.

Evaluation - Coggle Diagram

- - - - comprehensibility of error messages
    - - expert user's steady-state level of performance
      - speed of performance
  - - - Same results if tests were to be repeated?
      - Great inter-individual differences
      - Significance tests and estimation of confidence intervals
    - - Does the test actually measure what it is supposed to measure?
      - Requires methodological understanding and common sense
      - Wrong users, wrong tasks, not including time constrains and social influences
      - Avoid confounding
  - - - Sample
        
        representative
        
        seniority and experience
        
        Splitting in terms of sessions and tasks: novice vs. expert users
      - Sampling and bias
        
        Between vs within-subjects testing
        
        sample size
        
        individual variation
        
        skill transfer
      - matched vs. random assignment
      - Experimenters
        
        knowledge of the test method
        
        knowledge of the application and its user interface
      - Ethical considerations
        
        Pressure to perform
        
        Feeling inadequate or stupid
        
        Concerned about exhibiting ignorance
    - - Formative = e.g. thinking aloud; summative = e.g. measurement
    - - Convenience vs. representativeness
      - Comprehensibility of instructions
      - Adjustment of questionnaires
      - Timing of the session
      - Inconsistencies or weaknesses in the definitions
      - Refine experimental procedure
  - - - costly, not possible to test real-world problems
      - In a laboratory, researchers can record user activity
        
        while they fulfil predefined tasks for later analysis [6], and they
        
        can control other irrelevant variables [3]
    - - as users move around in
        
        field studies, data collection and conditions are difficult to
        
        control [1]. It can also be challenging to collect data in a precise
        
        and timely manner [7].