 |
ERCIM News No.46, July 2001 [contents]
|
by Wolfgang Dzida and Regine Freitag
Research at GMD involves trying to find an answer to the question: “Is a usability laboratory essential for usability testing?” Usability testing is taken as part of a comprehensive service accompanying the entire usability engineering process.
Two cases of investigations are typical for usability testing:
Investigations without a Usability Laboratory — when is this possible?
For every critical task the usability assessor must analyse the user’s efficiency and effectiveness in performing this task. Having a clear understanding of the flow of work, both user and assessor are able to interpret an encountered problem. If it is immediately and indisputably clear what is causing the problem, then it suffices to observe or inspect the situation and document it appropriately. Employing a usability laboratory would deliver the same result but with much higher costs. Hence, there should be good reasons when using such a laboratory.
Investigations with the Aid of a Usability Laboratory — when is this necessary?
The previously mentioned types of investigation require specific methodological approaches while using the equipment of a usability laboratory. In the first case the investigation is governed by a hypothesis the assessor aims to verify, but it is yet unclear whether the assumption concerning the cause of the usage problem is correct. In the second case it is not even clear whether a usage problem exists at all, and this must be explored empirically.
Hypothesis-driven Investigation
Usually, the user causes the usability assessor to have a closer look at a usage problem that is hard to work around. A plausible hypothesis as to the possible causes of the trouble is generated. The first step in forming a hypothesis is to identify the ‘mismatch’ between the user’s intention in a particular task, and the designer’s intention in providing the appropriate means for that task.
Scrutinising the mismatch requires a closer look at the case. However, the case is temporary. Should one ask the user to reproduce the critical situation again and again? Since the observational situation is also a social situation between people, this could mean expecting too much from the user. The short-lived interaction with the system can be more effectively captured by means of a video camera, thus allowing the assessor to repeatedly watch the episode afterwards in order to analyse and comment on it. Interpreting a mismatch is done by analysing the intentions of the user and contrasting them with the intentions of the designer. The GMD laboratory applies a specific template for coding the intentions in a semi-formal way, so as to precisely and completely identify the sources of mismatch.
Exploring Task Performance
Exploring task performance is especially interesting during usability prototyping to support requirements development. This approach is also referred to as explorative prototyping. A typical case of mismatch when using a new system or prototype is called opportunistic use (Carroll, 1992). This may happen, if, for example, the user misses an appropriate ‘affordance’ (Norman, 1989). The user then does not understand the designer’s intention and starts exploring some plausible opportunities.
In the case that an opportunistic strategy fails, the user runs into a usage problem. Because it can hardly be foreseen when this situation will occur, the whole session should be recorded with the aid of a usability laboratory. The laboratory primarily serves for the storage of observational data. Furthermore, the data can be exploited in order to rapidly extract and analyse the most interesting episodes out of the mass of recorded data.
The aim of exploration is to observe different users and study how they find their individual ways in conducting their tasks. Although the designer of the product has implemented an ideal way, it is unlikely that each user will intuitively follow this path. There is rarely ‘one best way’ of doing things. A usability laboratory is well equipped to record the variety of approaches employed by different users. The designer himself can watch the recorded situations, can listen to the user’s comments and will be well prepared for a face-to-face meeting with participating users.
Usually, an investigation of interactive prototypes takes place in a usability laboratory only when the prototype has achieved a fairly advanced stage of development. Most of the preceding iterative improvement steps can be done in a more heuristic manner.
Equipment of the Semi-portable Laboratory
Link:
http://borneo.gmd.de/~freitag/
Please contact:
Wolfgang Dzida and Regine Freitag — GMD
Tel: +49 30 61 62 57 81
E-mail: wolfgang@dzida.de, regine.freitag@gmd.de