Continuous Interaction and Manual Control

by Gavin Doherty

---

Under the auspices of the TMR project ‘Theory and Application of Continuous Interaction Techniques’ (TACIT), the author has been looking at possible applications of classical manual control theory to support the design of the ‘continuous’ interfaces present in modern interactive systems.

As interfaces evolve, interaction techniques increasingly involve a much more continuous form of interaction with the user, over both human-to-computer (input) computer-to-human (output) channels. Such interaction could involve gestures, speech and animation in addition to more ‘conventional’ interaction via mouse or keyboard. This poses a problem for the design of interactive systems as it becomes increasingly necessary to consider interactions occurring over an interval, in continuous time. This issue is the focus of the TACIT project, the aim of which is to develop theories (including mathematically based models) and methodologies for the design of interfaces in which continuous interaction techniques are used. The main TACIT partners are Department of Psychology, University of Sheffield; DFKI - Deutsches Forschungs-zentrum für Künstliche Intelligenz GmbH, Saarbrücken; Department of Industrial Engineering, University of Parma; IT Human Computer Interaction Group, University of York; ICS-FORTH; Laboratoire CLIPS-IMAG, Grenoble and CLRC. The project runs to the end of 2001.

Manual Control Theory

One body of work the project has been looking at to aid in the analysis of such interfaces is manual control theory. The theory developed out of the efforts of feedback control engineers after WWII, who required models of human performance for continuous tasks such as tracking with anti-aircraft guns. This seems to be an area worth exploring, firstly since it is generally concerned with systems which are controlled continuously by the user, although discrete time analogues of the various models exist. Secondly, it is an approach which models both system and user and hence is compatible with research efforts on ‘syndetic’ models, in which aspects of both system and user are specified within the same framework. Thirdly, it is an approach where continuous mathematics is used to describe functions of time. Finally, it is a theory which has been validated with respect to experimental data and applied extensively within domains such as avionics.

The premise of manual control theory is that for certain tasks, the performance of the human operator can be well approximated by a describing function, much as an inanimate controller would be. Hence, in the literature frequency domain representations of behaviour in continuous time are applied. Two of the main classes of system modeled by the theory are compensatory and pursuit systems. A system where only the error signal is available to the human operator is a compensatory system. A system where both the target and current output are available is called a pursuit system. In many pursuit systems the user can also see a portion of the input in advance; such tasks are called preview tasks.

A simple and widely used model is the crossover model, which has two main parameters, a gain and a time delay. Even with this simple model we can investigate some quite interesting phenomena. For example consider a compensatory system with a certain delay, if we have a low gain, then the system will move only slowly towards the target, and hence will seem sluggish. Alternatively if the gain is very high, then the system is very likely to overshoot the target, requiring an adjustment in the opposite direction, which may in turn overshoot, and so on. This is known as oscillatory behaviour. Many more detailed models have also been developed; there are anthropo-morphic models which have a cognitive or physiological basis. For example the ‘structural model’ attempts to reflect the structure of the human, with central nervous system, neuromuscular and vestibular components. Alternatively there is the optimal control approach, where algorithmic models which very closely match empirical data are used, but which do not have any direct relationship or explanation in terms of human neural and cognitive architecture.

Applying the Theory

With regard to potential areas of application of the theory, firstly, there is the possibility of applying the theory directly to interfaces such as those with continuous haptic interaction. Such models give us better estimates of human performance than simple HCI approaches such as the model human information processor of Card et. al., and also allow us to consider a wider range of tasks than rules such as Fitt’s law for selection. It might perhaps also be useful to develop some refined models for specific technologies and scenarios, such as gestural interfaces. Secondly there is the possibility of incorporating some part of the models into formal specification techniques. Thirdly it may help in the development of signal processing models to complement cognitive models being applied in the project, such as the Interacting Cognitive Subsystems (ICS) model.

The TACIT partners will be running a workshop entitled ‘Continuity in Human Computer Interaction’ at CHI 2000, 2-3 April 2000.

Links:

TACIT Home Page: http://kazan.cnuce.cnr.it/TACIT/
European Annual Manual Conference: http://www.wbmt.tudelft.nl/mms/annualmanual/

Please contact:

Gavin Doherty - CLRC
Tel: +44 1235 44 6738
E-mail: G.J.Doherty@rl.ac.uk