Heuristic Evaluation

Heuristic Evaluation is a discount (shortened) form of usability testing, which uses evaluators to test a device rather than real end users. Heuristics can be used in some circumstances when real users cannot be involved for practical or confidentiality reasons, and can be extremely useful in the early stages of evaluation to identify major usability problems with a device. This method involves a systematic evaluation of a device by assessing its compliance with a pre-determined heuristics, which are principles or ‘rules of thumb’ that a device should meet. The list of heuristics for any device will be informed by the user requirements collected during stage 1 and 2 of development and will reflect the factors that have been identified as necessary in a quality medical device.

 

These may include factors such as:

 

  • The device should provide feedback to the user at each stage of data input
  • The device should provide a summary of inputted parameters to the user at the time of prompting them to proceed with operation.
  • Instructions for use of the device should be visible or easily retrievable whenever appropriate.
  • Users should be informed about what is going on with the system through appropriate feedback and display of information

 

The evaluators then evaluate the device against the list and decide whether it complies with each of the heuristics (and if so, to what extent) or whether it does not.

This method developed in the field of human-computer interaction research and is most suited to the evaluation of computer-based medical devices and their user interfaces. Heuristic evaluation does not seek to replace a full evaluation with a representative sample of ‘real’ target users, rather it is best used early on in evaluation to identify major, or more obvious usability problems. Identifying and fixing these types of problems early on means that a full user evaluation can be performed on a better quality prototype where the focus can be on more subtle usability issues. For example, if a full user evaluation is being conducted on a device with a small number of glaring usability issues it is likely that the users will focus on these during the evaluation and may not notice the less obvious problems, problems that may be equally as important. When considering this method developers should question whether it is appropriate to use evaluators without the medical experience (either as a patient or healthcare professional) of the target users. However, this technique could have a place in medical device development as an initial usability test that could be performed in-house early on to identify and fix initial problems before performing an extensive usability test on a later prototype.