ISEE - A Framework for the Evolution and Analysis of Recurrent Neural Networks for Embodied Agents

by Martin Hülse, Steffen Wischmann, and Keyan Zahedi

Complex phenomena like bistability, periodic, quasi-periodic and chaotic attractors can already be observed in small artificial recurrent neural networks. How can such a rich reservoir of emergent properties of neural networks be applied to challenging applications?

We utilize the framework of Evolutionary Robotics (ER) and Artificial Life (AL) to evolve recurrent neural networks (RNNs), which enable robot agents to act successfully in an open and changing environment while mastering a given task. The combination of ER and AL allows the emergence of new control strategies within the sensorimotor loop, as no a priori knowledge is used to bias a possible solution. Hence, it serves as a source of inspiration for research into basic neural structures and general principles of neural signal processing systems, which may find applications beyond robotics.

Using a special evolutionary algorithm, the ENS3 (evolution of neural systems by stochastic synthesis) implemented as part of the ISEE package (integrated structural evolution environment, see figure), recurrent neural network structures of the general type for robot control have been developed. The structure and size of RNNs to evolve are open to the evolutionary process, and parameters like synaptic strengths are optimized simultaneously.

Evolution environment.

The ENS3 can also be utilized to extend or couple existing RNNs to achieve additional functionality. In this way, different behavioural (sub-)functionality can be nonlinearly integrated in one network solving a global robot task. On the one hand, this strategy allows an incremental evolution of complex control structures solving robot tasks, including increasing numbers of subtasks and their effective coordination. On the other hand, such incrementally evolved RNNs give us a wide variety of empirical set-ups with which to investigate multifunctionality and robust behaviour changes in complex systems provided by nonlinear coupled neural systems.

The versatility of our proposed evolutionary environment is sketched in the figure, where some sample applications are shown. A co-evolution strategy was applied for the ring-shaped robot micro.eva, where five moveable arms inside the ring must maintain a rotation of the whole body on two passive rollers. Each arm was considered as an autonomous agent. Hence, the neural control for each arm was evolved in parallel but in separated populations. In one result, division of labour could be observed. The same evolutionary strategy was successfully applied to evolve neuro-controllers for the Sony Aibo robot. Here, three different submodules for controlling the forelegs, the hind legs and the coordination of the legs were evolved in parallel to fulfil stable and fast walking patterns.

The wheel-driven robot Do:Little is used for experiments in swarm robotics. We used our evolutionary environment to either evolve one controller, which is then homogeneously distributed within a group of up to 150 robots, or for evolving in parallel a heterogeneous set of RNNs, where each individual gets a different RNN. In this way, we evolved RNNs for solving cooperative tasks, such as collecting energy and efficient exploration of the environment.

However, the evolutionary environment is not limited to the optimization of neural controllers. We also successfully evolved morphological parameters of a bipedal robot and its neural control at the same time. As a result, we achieved bipedal walking on flat surface with minimal control and high energy efficiency, using its passive dynamics for walking. It only needs to compensate the loss of energy due to friction and heel strike, which is solved by a comparably small neural controller.

Finally, in the figure one can see a snapshot of some tools that have been integrated into the software package. With tools providing visualization of RNNs and their neural activation during robot-environment interaction, one can gain a deeper understanding of the relationship between neural dynamics and observed behaviour. The dynamic properties of RNNs can also be investigated from a dynamic systems perspective. For instance, one can plot bifurcation diagrams or isoperiodic plots in order to identify attractors of dynamic systems such as RNNs.

With our software architecture it is possible to evolve different behaviours for arbitrary robotic platforms. One can either use fast simulations or even physical robots for the evolutionary process, as well as for a detailed analysis of control dynamics during robot-environment interactions. Although we mainly applied the ISEE package to robotic applications, it is not limited to the optimization of morphology and control. It can also be applied to various optimization problems unrelated to robotics, such as function approximation, parameter optimization and so on. The strength of the ISEE package is therefore its versatility and its powerful analysis tools.

Link:
http://www.ais.fraunhofer.de/INDY

Please contact:
Keyan Zahedi, Fraunhofer Institute for Autonomous Intelligent Systems - AIS, Fraunhofer ICT Group, Germany
Tel: +49 2241 14 2253
E-mail: keyan.zahediais.fraunhofer.de