Please enable JavaScript.

Coggle requires JavaScript to display documents.

EVALUATION - Coggle Diagram

- - - - About Usability Testing
        
        Emphasizes how usable a product is.
        
        Enables designers to control what users do and allows them to control the environmental and social influences that might impact the user’s performance.
        
        Test whether the product being developed is usable by the intended user in order to achieve the tasks for which it was designed and whether users are satisfied with their experience.
      - Methods, Tasks and Users
        
        Methods
        
        A combination of methods is often used to collect data
        
        User satisfaction questionnaire
        
        Collecting data: video recordings of the users, facial expressions, logged keystrokes, and mouse and other movements
        
        Ask participants to describe what they are thinking and doing out loud while carrying out tasks
        
        Structured or semistructured interviews
        
        Tasks
        
        Searching for information
        
        Reading different typefaces
        
        Navigating through different menus
        
        Uploading apps
        
        Users
        
        Number of users completing a task successfully
        
        Time to complete a task
        
        Time to complete a task after a specified time away from the product
        
        Number and type of errors per task
        
        Number of errors per unit of time
        
        Number of navigations to online help or manuals
        
        Number of users making a particular error
      - Labs and Equipments
        
        Mobile usability equipment: video cameras, laptops, eye-tracking devices, and other measuring equipment that can be set up temporarily in an office or other space, converting it into a
        makeshift usability laboratory
        
        lab-in-a-box or lab-in-a-suitcase
        
        custom-built usability labs: a main testing lab with recording equipment, an observation room where the designers can watch what is going on and how the data collected is being analyzed, a reception area where users can wait, a storage area, a viewing room for observers
    - - Hypotheses Testing
        
        Examining a relationship
        between two variables
        
        An independent variable is
        what the researcher manipulates
        
        Setting up a hypothesis to test
        
        An dependent variable
        
        The null hypothesis
        
        The alternative hypothesis
        
        Setting up hyptheses
        
        When a difference is specified but not what it will be, it is called a two-tailed hypothesis.
        
        Alternatively, the hypothesis can be stated in terms of one effect. This is called a one-tailed hypothesis
        
        A one-tailed hypothesis would be preferred if there was a strong reason to believe it to be the case.
        
        A two-tailed hypothesis would be chosen if there was no reason or theory that could be used to support the case that the predicted effect would go one way or the other.
      - Experimental Design
        
        Different-participant design
        (between subjects)
        
        A single group of participants is allocated randomly to each of the experimental conditions
        
        Advantage: no ordering or training effects caused by the influence of participants’ experience upon tasks
        
        Disadvantage: large numbers of participants are needed
        
        Same-participant design
        (within subjects)
        
        All participants perform in all conditions so that only half the number of participants is needed
        
        Advantage: Eliminates individual differences between experimental conditions.
        
        Disadvantage: Need to counterbalance to avoid ordering effects.
        
        Matched-pairs design
        (pairwise)
        
        Participants are matched in pairs based on certain user characteristics such as expertise and gender. Each pair is then randomly allocated to each experimental condition.
        
        Advantage: No order effects. The effects of individual differences are reduced.
        
        Disadvantage: Can never be sure that subjects are matched across variables that might affect performance.
      - Statistics: t-tests
        
        t-tests are the most widely used statistical test in HCI and related fields, such as psychology.
        
        The scores are used to compute the means and standard deviations
        
        The standard deviation is a statistical measure of the spread or variability around the mean.
        
        The t-test uses a simple equation to test the significance of the difference between the means for the two conditions.
        
        If they are significantly different from each other, we can reject the null hypothesis and in so doing infer that the alternative hypothesis holds.
  - - - In-the-Wild Studies
        
        New technologies are deployed and evaluated in natural settings
        
        Researchers often explore new technological possibilities that can change and even disrupt participants’ behavior
        
        Opportunities are created, interventions are installed, and different ways of behaving are encouraged
        
        A key concern is how people react, change, and integrate the technology into their everyday lives.
      - Other Perspectives
        
        Field studies may also be conducted where a behavior of interest to the researchers reveals itself only after using a particular type of software for a long time
      - Field studies are done in natural settings
      - Product Design
        
        Identify opportunities for new technology
        
        Determine design requirements
        
        Decide how best to introduce new technology
        
        Evaluate technology in use
  - - - Heuristic Evaluation
        
        About
        
        Researchers, guided by a set of usability principles known as heuristics
        
        Evaluate whether user-interface elements, such as dialog boxes, menus, navigation structure, online help, and so on, conform to tried-and-tested principles.
        
        These heuristics closely resemble high-level design principles
        
        Revised version (2014) of
        Nielsen’s original heuristics
        
        Visibility of System Status
        
        Match Between System and the Real World
        
        User Control and Freedom
        
        Consistency and Standards
        
        Error Prevention
        
        Recognition Rather Than Recall
        
        Flexibility and Efficiency of Use
        
        Aesthetic and Minimalist Design
        
        Help Users Recognize, Diagnose, and Recover from Errors
        
        Help and Documentation
        
        Doing Heuristic Evaluations
        
        Briefing session to tell experts what to do
        
        Evaluation period of 1-2 hours :
        
        Each expert works separately
        
        Take one pass to get a feel for the product
        
        Take a second pass to focus on specific features
        
        Debriefing session in which experts work together to prioritize problems
        
        Turning Design Guidelines, Principles,
        and Golden Rules into Heuristics
        
        The “eight golden rules.”
        
        Strive for consistency.
        
        Seek universal usability.
        
        Offer informative feedback.
        
        Design dialogs to yield closure.
        
        Prevent errors.
        
        Permit easy reversal of actions.
        
        Keep users in control.
        
        Reduce short-term memory load
        
        Granolle's suggested questions
        
        Does the application include a visible title page, section or site?
        
        Does the user alwaysknow where they are located?
        
        Does the user always know what the system or application is doing?
        
        Are the links clearly defined?
        
        Can all actions be visualized directly (i.e.,no other actions are required)?
        
        Heuristics for websites
        
        Clarity
        
        Make the system as clear, concise, and meaningful as possible for the intended audience.
        
        Minimize Unnecessary Complexity
        and Cognitive Load
        
        Make the system as simple as possible for users to accomplish their tasks.
        
        Provide Users with Context
        
        Interfaces should provide users with a sense of context in time and space.
        
        Promote a Pleasurable and
        Positive User Experience
        
        The user should be treated with respect, and the design should be aesthetically pleasing and promote a pleasurable and rewarding experience.
        
        Advantages and Problems
        
        Few ethical and practical issues to consider because users not involved
        
        Can be difficult and expensive to find experts
        
        Best experts have knowledge of application domain and users
        
        Biggest problems:
        
        Important problems may get missed
        
        Many trivial problems are often identified, such as false alarms
        
        Experts have biases
      - Walk-Throughs
        
        Cognitive Walk-Throughs
        
        The characteristics of typical users are identified and documented, and sample tasks are developed that focus on the aspects of the design to be evaluated. A description, mock-up, or prototype of the interface to be developed is also produced, along with a clear sequence of the actions needed for the users to complete the task.
        
        A designer and one or more UX researchers come together to do the analysis.
        
        3.The UX researchers walk through the
        action sequences for each task, placing
        it within the context of a typical scenario.
        
        Will the correct action be sufficiently evident to the user?
        
        Will the user notice that the correct action is available?
        
        Will the user associate and interpret the response from the action correctly?
        
        As the walk-through is being done, a record of critical information is compiled.
        
        The design is then revised to fix the problems presented. Before making the fix, insights derived from the walk-through are often checked by testing them with real users.
        
        Pluralistic Walk-Throughs
        
        Variation on the cognitive walkthrough theme
        
        Performed by a carefully managed team
        
        The panel of experts begins by working separately
        
        This is followed by a managed discussion that leads to agreed decisions
        
        The approach lends itself well to participatory design
        
        Also other adaptations of basic cognitive walkthroughs
        
        Another type of well-established walk-through
    - - Web Analytics
        
        About
        
        A form of interaction logging that analyzes users’ activities on website
        
        Designers use the analysis to improve their designs
        
        When designs don’t meet users’ needs, they will not return to the site
        
        They become one-time users
        
        Goals of using
        
        Web analytics enable designers to track the activities of users on their site
        
        They can see how many people come to the site, how long they stay, and where they go
        
        Web analytics offer designers the “big picture” about how their site performs based on user activity
        
        One of the most
        well-known analytics
        
        Google Analytics
        
        Types
        
        On-site analytics are used by website owners to measure visitor behavior.
        
        Off-site analytics measure a website’s visibility and potential to acquire an audience on the Internet regardless of who owns the website.
      - A/B Testing
        
        A large-scale experiment
        
        Offers another way to evaluate a website, application of app running on a mobile device
        
        Often used for evaluating changes in design on social media applications
        
        Compares how two groups of users perform on two versions of a design
        
        Can involve thousands of users
        
        May create ethical dilemmas if users don’t know they are part of the test
        
        Care is needed to ensure that other issues are not affecting users’ behavior
    - - Fitts’ Law
        
        Fitts’ Law predicts that the time to point at an object using a device is a function of the distance from the target object and the object’s size
        
        The further away and the smaller the object, the longer the time to locate it and point to it
        
        It is particularly useful for determining where on a screen to position an object
        
        Fitts’ Law is useful for evaluating systems for which the time to locate an object is important, for example, smartphones, handhelds, and mobile devices
      - Provide a way of evaluating products or designs without directly involving users
      - Less expensive than user testing
      - Usefulness limited to systems with predictable tasks, for example, voicemail systems, smartphones, and dedicated mobile devices
      - Based on expert error-free behavior