A Syndetic Approach to Human Computer Interfaces
by Giorgio Faconti and Mieke Massink
With the advent of new multimedia interaction technologies, the challenge is to demonstrate their usability. The role of human cognitive abilities is becoming increasingly important. It must be possible to reason about usability long before a system is built or even designed. In the early stages, the most important issues are requirement capturing and the development of an abstract design specification. As no prototypes of the human computer interface are available at that point, no experimental information can be obtained about its usability. A theoretical model of the cognitive aspects of human information processing can help to reason about the requirements for the interface. Unfortunately, traditional software engineering and system design methods do not offer the necessary support for this reasoning.
Our research attempts to fill this gap by expressing user and system representations in a common framework. The aim is to develop a theoretical background of cognition that provides a basis for arguing whether an interface is successful, discovering potential problems and suggesting solutions. Our approach does not aim at substituting existing methods of interface design but at achieving a qualitatively better initial design from which to begin an iterative design process.
In previous approaches, theoretical psychologically-based models have been used indirectly. We now directly adopt cognitive models within the specification process, as factors that affect usability depend on psychological and social properties of cognition and work, rather than on abstract mathematical models of programming semantics. Claims made through formal methods about a system's usability must, therefore, be grounded in some psychological or social theory.
Although in principle any cognitive theory could be adopted, we address Phil Barnard's Interacting Cognitive Subsystems (ICS), shown in the figure, in which a gesturing configuration is deployed. We formally model aspects of this theory so that it can be combined with a system specification following the syndetic modelling approach developed within the ESPRIT BRA AMODEUS project.
The 'Interacting Cognitive Subsystem' model with the gesturing configuration.
The word syndesis comes from the ancient Greek meaning to bring together. It conveys the idea of being able to reason about complex systems as a whole while keeping the capability of isolating and reasoning about their basic components. In our case, the syndetic model of an interactive system extends the formal model of its interface with the model of the cognitive resources needed to interact with the devices. The formal model of the system on its own provides few insights into the usability of its interface, while the formal model of ICS supports general claims about the user's cognitive processes but not about the effective use of cognitive resources in a given context. However, combining them in a syndetic model, we can reason about how cognitive resources are mapped onto the functionality of the system.
Currently, two different modelling approaches have been investigated: the first is based on the Modal Action Logic (MAL), the other on a functional data flow approach. In the first approach, the various components, such as interaction devices, system and user, are modelled as interactors object-like entities with an internal state, a presentation and actions that bring about changes to the state. The behaviour of each interactor can be formulated as an axiomatic model. In this way, both the user's cognitive resources and the behaviour of the interface can be described in the same framework, allowing for an integration of different models at a relatively high level of abstraction.
In the second approach, it is recognised that the structure of the cognitive ICS theory closely resembles a data flow oriented approach. The formal model and the more informal psychological model remain similar which is an advantage in multi-disciplinary discussions. The ICS concepts we address are the formal modelling of the continuous flow of information between subsystems and the transformation of these flows. Since the information flow is not always linear from the sensory subsystems to the effector subsystems, interesting problems concern the delay and synchronisation of information. The speed of information processing depends on the particular cognitive configuration, and the ability of the processes to operate on the available information. The delay observed between a stimulus and a response depends on whether a particular reaction has been proceduralised by the processes involved, or whether a level of reciprocal interchange of information or buffered processing of information is needed to interpret the available information. These differences are well-illustrated by the data flow approach and within this formalism we can formulate solutions for delay-related synchronisation problems. These solutions can be matched with results obtained from psychological experiments on human information handling.
The ICS model has been applied to the evaluation of gesture-based Human Computer Interfaces and of a more complex case of deitic reference employing gesture and speech. This work has shown that a formalisation of certain aspects of human behaviour is valuable in evaluating usability aspects based on initial formal designs of interfaces. Moreover, a more precise mathematical model of cognitive aspects can help to improve cognitive theory by raising new questions about its contents and application. These in turn can shed new insights into design issues for interactive systems. The formulation of the cognitive theory in a formal framework has the advantage that it can be easily introduced and explained to the computer science community.
Please contact:
Giorgio Faconti or Mieke Massink - CNR-CNUCE
Tel: +39 50 593 241
E-mail: G.Faconti@cnuce.cnr.it
and M.Massink@guest.cnuce.cnr.it
