Textbook of Usability

A within-subjects design (also called repeated measures) is an experimental design in which each participant experiences all conditions of the independent variable. If comparing design A against design B, every participant uses both designs.

Advantages:

• Controls for individual differences: each participant serves as their own control
• Requires fewer participants for the same statistical power (typically 2–3 times fewer than between-subjects)
• Directly measures within-person differences

Disadvantages:

• Learning effects: performance with the second design may be boosted by experience with the first
• Fatigue effects: participants may perform worse on later conditions because they are tired
• Carryover effects: exposure to one design may change how participants approach the next

Mitigation: counterbalancing. Half the participants use design A first and B second; the other half use B first and A second. For more than two conditions, Latin square designs ensure each condition appears in each position equally often.

Within-subjects is usually the most efficient choice for comparing two designs in usability experiments. It becomes inappropriate when exposure to one condition would contaminate the other, for example, comparing onboarding flows, where seeing one condition teaches the user what to do in the next.