Evolving generalist controllers to handle a wide range of morphological variations

Based on a conference paper:

Triebold, C., Yaman, A. Evolving generalist controllers to handle a wide range of morphological variations​. In Proceedings of the 2024 Genetic and Evolutionary Computation Conference (GECCO ’24) 2024.

(*Bachelor Thesis 2023, received Amsterdam AI Thesis Award*)

While neuro-evolution has demonstrated its effectiveness in learning neural network controllers on a range of tasks, the study of their robustness and generalizability remained limited. This has crucial impact in control tasks, especially dealing with morphological variations/changes in robotics for maintaining reliable performance.

This work proposes an algorithm that can produce robust and generalizable controllers by introducing morphological variations during the evolutionary process. As a result, the evolved controllers are able to handle a wide range of morphological variations with sufficiently well performance. We provide an extensive analysis on the trade-off between the specialist and generalist evolved controllers produced by our algorithm.

A generalist controller on CartPole

Key contributions

An evolutionary approach for producing generalist controllers: we proposed an evolutionary training approach that introduces morphological variations during the evolutionary process that leads to the generalization of the evolved artificial neural network based controllers. Firstly, we show that, indeed, introducing variation during training can lead to generalization property in terms of sufficiently controlling a range of morphological variations. Furthermore, the way that these variations are introduced can have an impact in the level of generalization. For instance, in our results, introducing variations based on small increments led to the most generalization.

A generalist controller on Bipedal Walker

Trade-off of specialization vs. generalization: we provide an extensive analysis on the results and show that there is a clear trade-off between specialization vs generalization. Our results supported the notion that, generalist controllers are better at controlling a wide range of morphologies, however, they come with the cost on a reduced performance on particular morphologies relative to the specialists.

A generalist controller on Ant

Evolutionary branching and ensembles: another interesting feature of our algorithm is that it can generate set of generalist controllers by partitioning the morphological variation space if one single generalist controller cannot be discovered to control the whole morphological variation space sufficiently well. As a result, the algorithm can output a set of controller that can handle certain ranges of morphological variations that can also provide insights into the task.

A generalist controller on Walker2D

Impact and implications

The contributions presented in this paper can potentially inspire new research directions in artificial intelligence and robotics. This work expands upon the, currently, limited examination of the generalizability and robustness of neural networks generated via neuro-evolution. It also introduces a novel approach for enhancing their generalisation capabilities. Our findings can inspire future research into the adaptation and development of additional techniques aimed at advancing neural networks obtained through neuro-evolution.