Usability Testing

Basic Characteristics

• Usability testing is a method to evaluate the usability of a product in an objective and comprehensible way.
• Users or potential future user are asked to perform realistic tasks by using the product or a prototype of it.
• Testing happens late in the development process. The product must be developped to an extent that performing a realistic task is possible
• Fair testing is assumed. During the task performance there is no (or very limited) assistance by the researcher. One of the research questions is: "Can users perform and complete the task wihtout assistance?"

Procedure to prepare and implement the testing

Preparation

• Define test objectives
  • Scope: Which part of the product is subject to evaluation?
  • Criterion: What are the evaluation criteria you want to check for?
• Define suitable task(s)
  • based on realistic scenarios, formulated in the language of the users
  • write instructions to explain the task to test persons
• Test criteria - e.g.
  • performance: e.g. speed, accuracy, number of errors, task completion
  • workload, mental/physical/temporal load
  • attention, situational awareness, mental information processing

Define target users and recruit participants

• define target users
  • potential or real users
  • based on personas
  • subject matter experts
• define sample size/number of participants
  • small samples (3-4 part.) allow first hypotheses and problem detection
  • larger samples allow high sensitivity and thorough detection of usability problems

Plan and prepare location and infrastructure

• Where does it happen? Lab, Field, Online
• Install the prototype/software/product/service
• Prepare data collection methods and infrastructure

e.g. sensors, footage, thinking aloud recording, eye-gaze-recording, surveys, etc.

• Prepare instructions to explain tasks and testing procedure to test persons.

Implementation of the testing

• Scheduling: Recruit and invite test persons.
• Briefing: Give instruction to the test persons.
• Start the test and the data recording.
• Wrap-Up:
  • Debriefing
  • Backup the data