Textbook of Usability

The evidence hierarchy for design is a framework for organising the sources of knowledge that support design decisions, analogous to the evidence hierarchies used in evidence-based medicine. Different levels of evidence offer different strengths, and strong design decisions draw on multiple levels that converge on the same conclusion.

The levels, from weakest to strongest (Chapter 20):

1. Expert opinion and intuition — a designer's judgement shaped by experience. Valuable but subject to bias and difficult to replicate.
2. Evolved practice — design conventions that have survived cultural selection over centuries. Observational but with huge sample size and strong selection pressure.
3. Design heuristics and expert consensus — principles like Nielsen's 10 heuristics, distilled from research and practice.
4. Observational studies — usability tests, field studies, surveys, and analytics measuring actual performance.
5. Predictive models — Fitts's Law, GOMS, cognitive load theory: models derived from controlled experiments that generalise to new designs.
6. Controlled experiments — randomised experiments isolating causal effects of specific design features.

No single level is sufficient. Controlled experiments provide the strongest causal evidence but cannot cover every design decision. Predictive models generalise well but address only specific aspects. Heuristics provide broad coverage but lack specificity. Evolved practice offers time-tested guidance but may not apply to novel contexts.

The strongest design arguments draw on convergent evidence from multiple levels. The case for larger touch targets, for example, is supported by Fitts's Law (model), controlled experiments, observational usability testing, evolved platform guidelines, and heuristic principles — five independent sources pointing to the same conclusion.