There are several methods to evaluate a design:
Usability studies
- This could be informal, watch and learn or formal usability labs.
- They offer good learning and uncover quirks, bugs or false assumtions.
- it is not the same as user’s using it to perform real work in real environment
- difficult to compare alternatives
- experimental bias
Surveys
- quickly gets feedback from a large number of users
- relatively easy to compare alternatives
- No need to build prototype, a screenshot or mockup will do.
- There is a difference between what people say they’ll do and what they actually do.
Focus groups
- gather a small group of people to discuss a design or idea.
- could be difficult due to group dynamics or if the subject makes people uncomfortable
Feedback from Experts
- peer review
- dogfooding
- heuristic evaluation
Comparative experiments
- taking two or more distinct options and comparing their performance to each other.
- observe actual behaviour as opposed to self report (in surveys)
- it can be better than usability studies since it compares multiple alternatives.
- but you can’t observe people like in usability study.
Participant observation
- observe people in their actual work environment
Simulations
- Useful when alternatives can be mathematically evaluated against design goals
- Allows lots of alternatives to be compared
Finally
The method of evaluation to be used for the specific design goal depends on these (often conflicting) parameters:
- Reliability: could be reproduce the results
- Generalizability: applicability to larger set of people
- Realism: do the observations hold good in real world situations
- Comparison: can we easily compare different (new of existing) designs
- Work involved: the effort to get the feedback