Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 15: Analytical evaluation - Coggle Diagram
Chapter 15: Analytical evaluation
Aims
Describe
inspection
methods.
Describe how to perform
GOMS
and
Fitts
’ Law, and when to use them.
Explain
how to do doing heuristic evaluation and walkthroughs.
Discuss the
advantages
and
disadvantages
of analytical evaluation.
Show how
heuristic evaluation
can be adapted to evaluate different products.
Inspections
Several kinds.
Experts use their knowledge of users & technology to review software usability.
Expert critiques (crits) can be formal or informal reports.
Heuristic evaluation is a review guided by a set of heuristics.
Walkthroughs involve stepping through a pre-planned scenario noting potential problems.
Heuristic evaluation
Developed Jacob Nielsen in the early 1990s.
Based on heuristics distilled from an empirical analysis of 249 usability problems.
These heuristics have been revised for current technology.
Heuristics being developed for mobile devices, wearables, virtual worlds, etc.
Design guidelines form a basis for developing heuristics.
Nielsen’s heuristics
Match between system and real world.
User control and freedom.
Visibility of system status.
Consistency and standards.
Error prevention.
Help and documentation.
Recognition rather than recall.
Flexibility and efficiency of use.
Aesthetic and minimalist design.
Help users recognize, diagnose, recover from errors.
Discount evaluation
Heuristic evaluation is referred to as discount evaluation when 5 evaluators are used.
Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems.
3 stages for doing heuristic evaluation
Briefing session to tell experts what to do.
Evaluation period of 1-2 hours in which:
Each expert works separately;
Take one pass to get a feel for the product;
Take a second pass to focus on specific features.
Debriefing session in which experts work together to prioritize problems.
Advantages and problems
Few ethical & practical issues to consider because users not involved.
Can be difficult & expensive to find experts.
Best experts have knowledge of application domain & users.
Biggest problems:
Important problems may get missed;
Many trivial problems are often identified;
Experts have biases.
Cognitive walkthroughs
Expert is told the assumptions about user population, context of use, task details.
One of more experts walk through the design prototype with the scenario.
Designer presents an aspect of the design & usage scenarios.
Experts are guided by 3 questions.
Focus on ease of learning.
The 3 questions
Will the user notice that the correct action is available?
Will the user associate and interpret the response from the action correctly?
Will the correct action be sufficiently evident to the user?
=> As the experts work through the scenario they note problems.
Pluralistic walkthrough
"Pluralistic walkthroughs are another type of walkthrough in which users, developers and usability experts work together to step through a [task] scenario, discussing usability issues associated with dialog elements involved in the scenario steps"
Predictive models (GOMS, Key Storeke level Model)
Less expensive than user testing.
Usefulness limited to systems with predictable tasks - e.g., telephone answering systems, mobiles, cell phones,
etc.
Provide a way of evaluating products or designs without directly involving users.
Based on expert error-free behavior.
GOMS
Operators
- the cognitive processes & physical actions needed to attain the goals, e.g., decide which search engine to use.
Methods
- the procedures for accomplishing the goals, e.g., drag mouse over field, type in keywords, press the go button.
Goals
- the state the user wants to achieve e.g., find a website.
Selection rules
- decide which method to select when there is more than one. (p.707)
Keystroke level model
The keystroke model allows predictions to be made about how long it takes an expert user to perform a task.
GOMS has also been developed to provide a quantitative model - the keystroke level model.
Fitts’ Law (Fitts, 1954)
The further away & the smaller the object, the longer the time to locate it and point to it.
Fitts’ Law is useful for evaluating systems for which the time to locate an object is important, e.g., a cell phone, a handheld devices.
Fitts’ Law predicts that the time to point at an object using a device is a function of the distance from the target object & the object’s size.
Key points
Heuristic evaluation relatively easy to learn.
May miss key problems & identify false ones.
Relatively inexpensive because no users.
Predictive models are used to evaluate systems with predictable tasks such as telephones.
Expert evaluation: heuristic & walkthroughs.
GOMS, Keystroke Level Model, & Fitts’ Law predict expert, error-free performance.
