Textbook of Usability

A between-subjects design is an experimental design in which each participant experiences only one condition of the independent variable. If comparing design A against design B, one group of participants uses only A and a different group uses only B.

Advantages:

• No learning effects — participants encounter each design fresh
• No fatigue or carryover — previous conditions don't influence current ones
• Simpler procedure — each participant does a single session

Disadvantages:

• Individual differences between groups may obscure the effect of interest
• Requires more participants — typically 2–3 times as many as within-subjects for the same power
• Random assignment is essential to ensure comparable groups

Mitigation: random assignment distributes individual differences across groups in expectation. Stratified random assignment can further balance groups on known characteristics (gender, experience level, age).

Between-subjects is necessary when exposure to one condition would contaminate another — the most common example is A/B testing of live web products, where each user sees only one version. It is also used when learning or practice effects would dominate within-subjects results, as in studies of first-time experience.