Textbook of Usability

A within-subjects design (also called repeated measures) is an experimental design in which each participant experiences all conditions of the independent variable. If comparing design A against design B, every participant uses both designs.

Advantages:

• Controls for individual differences — each participant serves as their own control
• Requires fewer participants for the same statistical power (typically 2–3 times fewer than between-subjects)
• Directly measures within-person differences

Disadvantages:

• Learning effects — performance with the second design may be boosted by experience with the first
• Fatigue effects — participants may perform worse on later conditions because they are tired
• Carryover effects — exposure to one design may change how participants approach the next

Mitigation: counterbalancing. Half the participants use design A first and B second; the other half use B first and A second. For more than two conditions, Latin square designs ensure each condition appears in each position equally often.

Within-subjects is usually the most efficient choice for comparing two designs in usability experiments. It becomes inappropriate when exposure to one condition would contaminate the other — for example, comparing onboarding flows, where seeing one condition teaches the user what to do in the next.