You test your product, so you know its usability. But maybe you don't. Because in user tests participants represent users, simulations represent products, and the lab represents their living room. And you ask them to do something they may not do spontaneously. As a consequence test findings might not be that representative for the real world situation. Paula Wellings, from adaptive path, wrote an excellent piece about the representativeness of user testing: usually the focus lies on prototype fidelity, but - especially for physical, connected, multimodal products - also environmental and social fidelity should be considered.What it really comes down to is that you look at each of the elements that make up human-product interaction, and consider whether what is used in the test is representative for the real world. Shackel's (1984) framework for human-machine systems (pictured above) consisted of four elements: the user, his/her task, the system, and the environment in which the interaction takes place. In this paper (free pre-print pdf) we extended Shackel's framework to match the current situation for electronic consumer products (see picture below). System became product, environment became context and we added these elements:

• 'other users', because most consumer products are not used by just one person.
• 'other products', because the majority of electronic consumer products is not used in isolation, but have to be connected to other products sooner or later.
• 'product service combinations' instead of just 'product' or 'system', because a lot of electronic consumer products feature a service that impacts the product significantly, e.g., a mobile phone and the subscription, iPod and iTunes.

For electronic consumer products, we consider 'the goal' or task to be a part of the user.

The framework can be used as a sort of 'checklist'. Take a look at the elements and relations in the framework to take stock of the most important use cases for your test, and consider the respresentativeness of the elements and the relationships between them. And whether that really matters. Because sometimes, to fulfill the goal of your usability test you may not need the full degree of representativeness.

[Illustration at top: Bob van Vliet]