Coupling Perception with Actions via Mirror Neurons

by Jiri Wiedermann

A recent discovery in the brains of primates, mirror neurons are special neurons that show activity both when a subject performs an action and when it observes the same action performed by itself or another (possibly conspecific) subject. Mirror neurons present a versatile self-learning mechanism for the coordination of perceptional and motor actions. They also seem to present a potential bridge between minds. An understanding of the coupling of perception with actions via the mirror neurons, and the management of the respective internal representations, has opened the road to elucidating the computational principles underlying human cognition.

Operating within the framework of long-term research into the theory and applications of neural nets and computational intelligence, a project on machine perception and embodied cognition commenced in 2003 at the Institute of Computer Science of Academy of Sciences of the Czech Republic. The immediate aim has been to elucidate the role of mirror neurons in sensorimotor control and the consequences for imitation learning; a more distant goal is the study of perception-motor coordination in relation to abstract concept formation, language development, understanding, acquisition and generation, and to human-like cognition in general.

The research was initiated by the original hypothesis on the role of mirror neurons in cognition. It has been conjectured that the experimentally observed properties of mirror neurons are only part of a larger associative mechanism that serves to complete cross-modal (perception and motor) information. In this way, the mirror neurons allow ‘virtual’ (off-line) information to be processed. This hypothesis could therefore explain the mechanisms behind sensorimotor coordination, imitation learning and thinking, since these activities are viewed as specific modes of work of the mirror neural net.

In order to support this conjecture, a computational model of an embodied cognitive agent incorporating the mirror neural net has been designed. The model consists of two neural nets called the mirror neural net and the cogitoid, respectively. Attached to these nets are so-called perception-motor units (PMUs). Each PMU delivers corresponding perception information to the mirror net and the cogitoid sends instructions for motor actions to the PMUs. Perception information is of two kinds: sensory, from the sensors delivering data from the environment, and proprioceptory, delivered by the sensors within each PMU and reporting the current state of that PMU. In the mirror net, perception information meets the instructions that have been issued for PMUs. The resulting stream of data is checked for consistency and completeness by comparing it with previous similar cases, and recovered if necessary. The completed information then proceeds to the cogitoid. Its task is to create concepts corresponding to frequently performed actions and to build associations among them that are used for selecting the next action and instructing the PMUs. The cogitoid was described in ERCIM News No. 53, 2003. The resulting neural architecture of an embodied cognitive agent is depicted in Figure 1. Note that the mirror net takes care of the grounding of concepts, while its associative properties provide 'situatedness' to the agent under circumstances when the perception or motor part of the multimodal information is missing.

The neural architecture of an embodied cognitive agent.

The first theoretical results from this formal model of an embodied cognitive agent are well in line with some current theories that, however, are not systematically based on a computational model and usually do not span the whole range of human cognition. This model leads to a clear distinction between data processing and cognitive tasks. It also allows definitions of various cognitive tasks in terms of interactive ‘games’ between a situated agent and its environment, or between several agents. Its algorithmic properties enable a plausible insight into mechanisms of the mind that were hitherto only anticipated. In particular, the coupling of perception with actions via a mirror net, its associative properties and the introduction of a mechanism for concept and association formation (the cogitoid) have opened the door to understanding the mechanism of understanding, the evolution of inter-agent communication leading to language emergence, and indeed to the development of a rudimentary mind. So far, the project is of a more speculative, rather than experimental nature, but it is assumed that in the near future, the practical potential of such an embodied neural architecture for imitation learning will be realised. The use of Khepera robots and their engagement in various imitation games is envisaged for such a purpose. This phase of the project relies on the involvement of PhD students from the Faculty of Mathematics and Physics at Charles University in Prague, and it is also expected that the Laboratory of Machine Perception at the Czech Technical University will be part of the collaboration. This research has been partially supported by GA CR grant No. 201/02/1456

Links:
See the related technical report V894-03.ps
by the author at http://www.cs.cas.cz/

Please contact:
Jiri Wiedermann, Institute of Computer Science, Academy of Sciences of the Czech Republic (CRCIM)
Tel: +4202 6605 3520
E-mail: jiri.wiedermann@cs.cas.cz