Why are Evolved Developing Organisms also Fault-Tolerant? Diego Federici1 and Tom Ziemke1 University of Sk¨ ovde, Sk¨ ovde, Sweden {diego.federici | tom.ziemke}@his.se

Abstract. It has been suggested that evolving developmental programs instead of direct genotype-phenotype mappings may increase the scalability of Genetic Algorithms. Many of these Artificial Embryogeny (AE) models have been proposed and their evolutionary properties are being investigated. One of these properties concerns the fault-tolerance of at least a particular class of AE, which models the development of artificial multicellular organisms. It has been shown that such AE evolves designs capable of recovering phenotypic faults during development, even if faulttolerance is not selected for during evolution. This type of adaptivity is clearly very interesting both for theoretical reasons and possible robotic applications. In this paper we provide empirical evidence collected from a multicellular AE model showing a subtle relationship between evolution and development. These results explain why developmental fault-tolerance necessarily emerges during evolution.

1

Introduction

That biological organisms display various levels of robustness is a well known fact. Waddington referred to this tendency to suppress phenotypic variation as canalization [1, 2]. Two types of canalization are distinguished. Genetic canalization describes the phenotypic resistance to alterations of the genotype (herein Mutational Robustness). Environmental canalization is instead the organism’s capacity to suppress external influences. The latter comprises fault-tolerance as the ability to recover from transient phenotypic faults during development. It is widely accepted that, when noise is present at the level of both the genotype and the phenotype, canalization emerges as an adaptive response under the influence of natural selection. This stabilizing selection captures the inclusive fitness advantage derived by robustness [3]. But is stabilizing selection strictly necessary for achieving robustness? (see also [4]). This question is of great interest both for theoretical reasons and for engineering purposes. The possibility to develop designs/algorithms which can autonomously recover from faults during operation, much like living systems, is clearly very appealing. The classical engineering approach to fault-tolerance is

functional redundancy, but biological organisms can also recover faults by homeostatic processes, such as self-healing and regeneration. Theoretically a perfectly regenerating individual (e.g. a robot) could continue to operate ad infinitum without external maintenance. A notable example is provided by the Hydra Oligactis. Hydras can regenerate any damaged or dead cell, and severed body parts can even reconstruct the complete organism [5]. Famous are also the limb of the salamander and the tail of the lizard, which can be regrown after being severed. Regeneration also takes place in the nervous system, as has been shown in recent studies [6, 7]. But testing all possible sources of faults can be prohibitively expensive, if not impossible. This simple fact hinders the construction of artificial systems which are too complex for mathematical analysis. These unfortunately include most evolutionary designs. Recently a few different models of multi-cellular development have been proposed as a possible solution to the scalability limitations of evolutionary computation [8–10]. An interesting property of these systems is that they have been shown to produce fault-tolerant designs in the absence of stabilizing selection [11, 10], in other words “for free”. We will refer to this property as emergent fault-tolerance. Emergent fault-tolerance may be caused by some intrinsic property of the multi-cellular model. For example, due to their distributed nature, artificial neural networks are known to show a graceful functional degradation in response to the loss of a few units or connections. On the other hand, in [10] it was shown that this does not appear to be the case for multi-cellular development since recovery from faults was shown to be present in evolved individuals but not random ones. In [12] emergent fault-tolerance was shown to appear during evolution after a few generations. This fact is quite interesting since it is in agreement with biological findings in selection experiments, where canalization is shown to evolve in few generations [13, 14]. It seems that the evolution of multi-cellular developing models shows a preference toward fault-tolerant individuals. Such a tendency can be exploited to produce designs with increased fault-tolerance even if during evolution only partial tests are carried on all possible sources of faults [15]. Similar results are also found in [10, 16]. But if it is not caused by stabilizing selection, why are evolved individuals fault-tolerant? In [12] it was shown that individuals displaying high mutational robustness also proved particularly fault-tolerant. It was then hypothesized that fault-tolerance is the developmental counterpart of mutational robustness. In fact with development phenotypes are constructed unfolding genotypes in time. A mutation can cause a phenotypic divergence in a phase, which can then be recovered in a following one. So, if with direct encoding mutational robustness can only be achieved by suppressing the phenotypic effects of mutations [17], with development variations can be expressed but still be neutral as long as they get corrected before the fitness test.

fault tolerance

A

fault

cell

Faulty

A Healthy

Mutated Development

mutational robustness

A

mutation

Fig. 1. Correlation between environmental and genetic canalization, Ahealthy is the original individual, Amutated its mutated offspring. Mutations that cause phenotypic effects can be recovered later (bottom row). In the same way faults occurring to Af aulty can be recovered as long as they are homologous to the typical divergence caused by mutations (top row). To be neutral to selection, divergence must be recovered before the fitness test.

The correlation between mutational and phenotypic robustness appears because, if individuals can recover phenotypic perturbations caused by mutations, they can also recover from similar phenotypic perturbations caused by faults (see Figure 1). In this paper, this hypothesis is put to the test and supporting is provided. First developing artificial organisms are evolved to match specific targets. Then the robustness of the best individuals is checked offline. Results show that recovery from both genotypic and phenotypic perturbations becomes stronger as the fitness test gets closer in time, therefore validating the initial hypothesis: fault-tolerance is the ontogenetic homolog of mutational robustness. These results highlight a subtle and indirect interaction between phylogeny and ontogeny. This interaction can both help to explain the evolutionary emergence of phenotypic robustness even in the absence of a direct evolutionary advantage (i.e. faults or noise), and be exploited to cheaply produce increasingly resistant artificial adaptive organisms (see also Conclusions).

2

Related Work

Typically introduced to increase the scalability and flexibility of evolutionary computation, several indirected encoding schemes have been proposed. These artificial embryogeny (AE, [18]) methods recursively construct the mature phenotype following the growth program defined in the genotype.

Since selection operates at the level of the phenotype, the relationship between the evolving genotype and its inclusive fitness is mediated by the development process. This indirect path may trigger complex gene-to-gene interactions, which are captured by the concept of the gene regulatory network (GRN). Since phenotypic maturation in AE is de facto a rewriting process, early models were based on grammar-based approaches in which the genotype defines the substitution rules which are repeatedly applied to the phenotype. Examples include the matrix rewriting scheme [19] and the cellular encoding [20]. Some models introduced additional contextual information in each rule definition [21, 22], so that phenotypic trait variations could be generated. Also, it is possible to implicitly define the grammar by means of an artificial GRN [23] and use the accumulated concentrations of simulated chemicals to modulate the characteristics of morphological constituents. In this direction, and inspired by cellular automata, a second approach is to evolve the rules by which cells alter their metabolism and duplicate. Cells are usually capable of sensing the presence of neighboring cells [24], releasing chemicals which diffuse in simulated 2D or 3D environments [25, 9], and moving and growing selective connections to neighboring cells [26]. Closely related to the one presented in this paper, the model proposed in [8] is based upon a fixed Cartesian 2D lattice, in which each cell occupies a given square. Artificial organisms are generated starting from a single cell. Every cell can replicate in the four cardinal directions taking the organism to maturation in a fixed number of development steps. All cells share the same genotype encoding the cell growth program (its regulatory network). In [8] the growth program is structured as a sequence of rules. Rules are activated by matching the local neighborhood of a given cell and trigger specific cell responses: duplication, death and cell-state change. In [9], the growth program is represented by a Boolean network. Cells belong to one of four different types and can release chemicals which undergo a simulated diffusion process. Specific evolutionary targets (2D patterns) were evolved and emergent self-healing dynamics were reported for the first time [11]. In [10] the previous model is extended with internal chemicals, which do not diffuse in the environment but are private to each cell. The growth program is encoded by a recursive neural network, and the organism’s genotype can contain several chromosomes, each one specifying a complete growth program. Individuals are initialized with a single chromosome which controls the entire development process. During evolution, additional chromosomes can be introduced by duplication (i.e. gene duplication [27]), each one being associated to a specific stage of development. By allowing several independent embryonal stages, this method proved capable of increasing overall evolvability in the evolution of specific 2D patterns, also showing a higher scalability then direct encoding. In this case too, emergent fault-tolerance was reported. In [8–10], fitness was based only on the topological displacement of cell in mature individuals. In [16] the AE model in [9] was used to produce a 2-bit multiplier capable of recovering transient phenotype faults. In [15] the AE model

Phenotype Space

development step

effect of a mutation

Zygote

region of stability effect of a fault

Development

Fig. 2. Illustration of the relation between mutational robustness and fault-tolerance. If a development program has a region of stability, as long as they do not take phenotypes outside of the region perturbations can be recovered, whatever their cause.

in [10] was used to evolve a regenerating spiking neuro-controller for simulated Khepera robots. These last results indicate the great potential that the evolution of complex fault-tolerant ontogenies can provide to the adaptive behavior community.

3

Methods

It has been hypothesized that fault-tolerance is a side-effect of the genotype’s mutational robustness (genetic canalization). Mutated genotypes could in fact produce identical mature organisms even if during development phenotypic divergence occurs. This as long as this phenotypic divergence is recovered/corrected before maturation, i.e. the fitness test, is reached (see Figure 2). The central point is that once a genotype evolves a means to recover from phenotypic divergence, it will do so whatever the cause: either if it is caused by a mutation affecting the growth program or if it is produced by similar transient faults. In either case, robustness should be stronger when the fitness test is closer in time, due to the fact that phenotypic divergence is only “apparent” to the selection mechanism at the time of the fitness test. We can therefore test the level of mutational and phenotypic robustness and see how it varies during development. To do so we will perturb evolved genotypes and phenotypes for a single step and let the developing individual recover for an

Best evolved individuals

Tao

Circle

Targets

12 steps

17 steps

Fig. 3. Evolutionary targets and best evolved individuals at steps 12 and 17. Above the Circle and below the Tao target. Fitness is the average resemblance to the target computed at steps 12 and 17

additional step. Divergence will be measured as the Hamming distance between the perturbed and non-perturbed individuals. One problem, however, is created by the fact that multicellular development starts from a single cell (the zygote) to reach maturation in a certain number of steps. During the initial expansion phase perturbations will have a different impact on development. To minimize this effect, it is possible to evolve individuals checking fitness at two different steps of development, in our case at step 12 and 17. It is reasonable to assume that during the 5 growth steps from 12 to 17 the phenotypes will be more stable. 3.1

Development Model

The AE model used in this paper is explained in detail in [10]. For clarity, a short summary of the model’s details follows. Phenotypes develop starting from a single cell placed in the center of a fixed size 2D rectangular array. Multicellular organisms reach maturation in a precise number of developmental steps. Cells replicate and can release simulated chemicals in intra-cellular space (cell metabolism).

Unlike other possible approaches [8], no predefined chemical gradients are present. These in fact offer a global contextual information which biases the evolution of development toward trivial solutions. Cell behavior in our model is governed by a growth program based on local variables, and represented by a simple recursive neural network (Morpher) with 4 hidden units. The Morpher input vector encodes the state of a particular cell (type and metabolism) and the types of the four neighboring cells in the North, West, South and East directions (NWSE). At each developmental step, under the control of the Morpher, existing active cells can change their own type, alter their metabolism and produce new cells. An active cell can also die or become passive. Each step, up to four new cells can be produced in any of the NWSE directions. In case of cell genesis, the mother cell specifies the daughter cells’ internal variables (type and metabolism) and whether they are active or passive. If necessary, existing cells are pushed sideways to create space for the new cells. When a cell is pushed outside the boundaries of the grid, it is permanently lost. Embryonal Stages The regulatory system controls gene expression over two orthogonal dimensions: time and space. Development with Embryonal Stages (DES) implements a direct mechanism of Neutral Complexification for the temporal dimension [10]. As development spans over several consecutive steps, the idea is to start evolution with a single growth program (chromosome/Morpher) which controls all the development steps. As evolution proceeds, a new chromosome can be added by gene duplication. The developmental steps are therefore partitioned into two groups/stages. The first, controlling the initial steps of embryogenesis, is associated with the old chromosome. The latter, completing growth, is associated with the new, identical, duplicated chromosome. Being exact copies, new chromosomes do not alter development, and are therefore neutral. But possible mutations can independently affect each duplicated gene. By unlocking the gene expression of different development phases, each chromosome can assume more specialized roles, de facto increasing the genotypic resolution around the area represented by the current mature phenotype. Overall, the effect is an increase in genotype-phenotype correlation leading to higher evolvability. In the simulations presented herein, only the chromosome controlling the latest stage is subjected to the evolutionary operators, while all other chromosomes remain fixed. 3.2

Evolutionary Details

Each cell in the mature phenotype is interpreted as a pixel, its color provided by the cell type (three possible). Fitness is proportional to the resemblance of an individual to the target pattern (see Figure 3) and is computed as the normalized Hamming distance to the target at steps 12 and 17. For fitness computation, dead

0.08

0.07

0.07

0.06

0.06

0.05

0.05

0.04

0.04

0.03

0.03

0.02

0.02

0.01

0.01 5

4

3 2 steps to fitness check

1

5

4

3 2 steps to fitness check

1

0 0.1

0.09

0.09

0.08

0.08

0.07

0.07

0.06

0.06

0.05

0.05

0.04

0.04

0.03

0.03

0.02

0.02

0.01

0.01

0

5

4

3 2 steps to fitness check

1

5

4 3 2 steps to fitness check

∆P caused by 10% fault

Circle

∆P caused by x2 mutation

0.09

0.08

0.1

Tao

0.1

0.09

0

∆P caused by x2 mutation

Fault Tolerance

1

∆P caused by 10% fault

Mutational Robustness

0.1

0

Fig. 4. Phenotypic divergence after the perturbation of a development step by means of two consecutive mutations (left) or the removal of each cell with a 10% probability (right). Circle (top) and Tao (bottom) individuals are allowed to recover one additional step. Robustness both to mutations and fault increases in temporal proximity of the fitness test. Averages and standard deviations over 100 perturbations, at each development step for each of the best individuals of the 40 populations.

cells are assigned the default type 0 (black color). Organisms grow in a 32x32 2D array starting from a single active cell in position (16,16), with type 1 and metabolism 0. Results are obtained from 20 independent populations for each target. Every population is composed of 400 individuals. The best 50 individuals are copied to the next generation and reproduce (elitism). Evolution comprises 2000 generations. 10% of the offspring are produced by crossover. The Morpher is modeled by a neural network with 7 inputs, 15 outputs and 4 hidden nodes. The genotype contains a floating point number for each of the 107 Morpher’s weights. Mutation takes each weight of the Morpher with a .1 probability and adds to it Gaussian noise with 0 mean and .1 variance.

4

Results

Fit individuals were produced in all the evolutionary runs. In the case of the Tao target the average fitness was 86 ± 2% with a maximum of 89%, while for the Circle it was 81 ± 3% with a maximum of 86% (see Figure 3). In Figure 4 we show how genotypic and phenotypic perturbations are recovered in a single ‘healing’ step by the best evolved individuals of each population. Perturbations are generated either by two consecutive mutations to the tested genotypes, or by removing each cell in the phenotype with a 10% probability. For both targets the phenotypic divergence ∆P decreases as the fitness test gets closer in time, both with genotype and phenotype perturbations. In the case of the Tao target, which is also the easier target to evolve, individuals appear more robust. Since in all cases robustness measures the ability to dampen divergence with a single ‘healing’ step, it appears that phenotypic perturbations are taken care of more effectively in the temporal proximity of the fitness test. That means, the closer the fitness test in time, the more robust the organisms appear to be. This indicates that fault-tolerance is not a normal tendency of development but it is directed toward the dampening of the observable phenotypic deviations, i.e. from the point of view of selection and fitness.

5

Conclusions

We have argued that environmental canalization (fault-tolerance) is the developmental homolog of genetic canalization (mutational robustness). Mutational robustness is the dampening of the observable1 phenotypic consequences of mutations. It has been shown to emerge spontaneously as an adaptive response to the evolutionary dynamics [12], as regular regions of the fitness landscape are more stable under natural selection [28]. For direct mappings from genotypes to phenotypes, lacking a temporal dimension, robustness can only be achieved by means of epistatic interactions (see for example [17]). With development on the other hand, phenotypic divergence can be recovered during growth. When a good correlation between ‘small genotypic changes’ to ‘small phenotypic changes’ is present, mutational robust developing organisms will also display fault-tolerance. In fact, ‘small genotypic changes’ are dampened by the evolutionary preference for regular regions of the fitness landscape, while ‘small phenotypic changes’ are recovered because they are homologous to the those (see also Figure 5). With this theory in place we can formulate a set of predictions: Evolvability Tests The presence of mutations pushes evolutionary-stable populations into genotype regions of stability. Since these regions are generated in 1

observable by means of the fitness function

G region of stability

map 2

map 1

P1

mutation

P2

fault

Fig. 5. An individual residing in a genotypic region of stability (top) will display a high degree of mutational robustness. Depending on the properties of the genotypephenotype mapping, the genotypic region of stability can project in phenotype spaces in different ways. In P1 small movements in genotype space produce small movements in phenotype space so that the region of stability is still, for the most, surrounding the individual’s phenotype. The opposite in the case of P2 . Faults occurring in P1 have a greater chance to be homologous to mutations in G and therefore be tolerated by development.

genotype space by dampening the effects mutations, in general we cannot expect that the G-region of stability projects nicely into phenotype space. For example, suppose you have a development system by which any small genotypic change can only cause big phenotypic consequences (a mapping based on a hashing function for example). In this case a mutationally robust individual may not display resistance to small faults, since small phenotypic variations are homologous to big leaps in genotype space, leaps which are unusual during evolution2 and will probably take the genotype out of its region of stability. In order for fault-tolerance to emerge, we must have a development system in which small phenotypic changes are homologous to the typical effects of mutations. The design of such systems is not a new problem in evolutionary computation since there is a wide consensus that they are associated with high levels of evolvability. We can use the emergence of fault-tolerance as an indication of evolvability. Since fault-tolerance is shown to emerge in few generations, evolvability can be 2

using the mutation operator to define the genotype space metric

sampled rapidly on a wide range of parameters before more extensive searches are conducted. Convergence and Robustness As shown in selection experiments [13, 14], populations undergoing a selective pressure for new characteristics also display less robustness. Evolution will in fact select those individuals capable of escaping the genotypic stability region, de facto pushing toward less robust genotypes. It might take several generations before robustness emerges again. To boost fault-tolerance, it could be possible to first evolve a suitable individual. In a second evolutionary phase, the old population would now evolve not toward the old target, but toward the best individual found in the first phase. By preferring younger individuals to old ones, or allowing selection with full replacement, the second phase will produce individuals with increased levels of canalization. Mutation Levels and Fault-Tolerance Since emergent fault-tolerance is a byproduct of mutational robustness, larger regions of stability are to be expected as the mutation rate is increased. This was in part validated by results contained in [12]. An increase in the mutation rate produced individuals with higher fault-resistance at the price of a decrease in overall fitness. Acknowledgments This work is part of the project “Integrating Cognition, Emotion and Autonomy” (IST-027819, www.his.se/icea), funded by the European Commission as part of the EC Cognitive Systems initiative.

References 1. Waddington, C.: Canalization of development and the inheritance of acquired characters. Nature 150 (1942) 563–565 2. Stearns, S.: Progress on canalization. Proc Natl Acad Sci USA (2002) 10229–30 3. Schmalhausen, I.: Factors of Evolution: The Theory of Stabilizing Selection. Univ. of Chicago Press; reprinted in 1986 (1949) 4. Siegal, M., Bergman, A.: Waddington’s canalization revisited: developmental stability and evolution. Proc Natl Acad Sci USA 99(16) (2002) 10528–32 5. Bode, P., Bode, H.: Formation of pattern in regenerating tissue pieces of hydra attenuata. i. head-body proportion regulation. Dev Biol 78(2) (1990) 484–496 6. Zhao, M., Momma, S., Delfani, K., Calren, M., Cassidy, R., C.B.Johansson, Brismar, H., Shupliankov, O., Frisen, J., Janson, A.: Evidence for neurogenesis in the adult mammalian substantia nigra. Proc Natl Acad Sci USA 100(13) (2003) 7925–30 7. Wu, D., Schneiderman, T., Burgett, J., Gokhale, P., Barthel, L., P.A.Raymond: Cones regenerate from retinal stem cells sequestered in the inner nuclear layer of adult goldfish retina. Invest Ophthalmol Vis Sci. 42(9) (2001) 2115–24 8. Bentley, P., Kumar, S.: Three ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In Banzhaf, W., al., eds.: Proc. of GECCO ‘99. (1999) 35–43

9. Miller, J.: Evolving developmental programs for adaptation, morphogenesys, and self-repair. In Banzhaf, W., Ziegler, J., Christaller, T., eds.: Proc. of the European Conference of Artificial Life, Lecture Notes in Artificial Intelligence, Vol. 2801. (2003) 256–265 10. Federici, D., K.Downing: Evolution and development of a multi-cellular organism: Scalability, resilience and neutral complexification. Artificial Life Journal (in press) 12:3 (2006) 11. Miller, J.: Evolving a self-repairing, self-regulating, french flag organism. In Deb, K., et al, eds.: Proc. of Genetic and Evolutionary Compuation, GECCO 2004. (2004) 129–139 12. Federici, D.: The evolutionary emergence of intrinsic regeneration in artificial developing organisms. In Ijspeert, A.J., Masuzawa, T., Kusumoto, S., eds.: Proc. of BioADIT 2006. (2006) 176–191 13. Kindred, B.: Selection for an invariant character, vibrissa number in the house mouse. v. selection on non-tabby segregants from tabby selection lines. Genetics 55(2) (1966) 365–373 14. Maynard-Smith, J., Sondhi, K.: The genetics of a pattern. Genetics 45(8) (1960) 1039–1050 15. Federici, D.: A regenerating spiking neural network. Neural Networks 18(5-6) (2005) 746–754 16. Liu, H., Miller, J., Tyrrel, A.: Intrinsic evolvable hardware implementation of a robust biological development model for digital systems. In: Proc. of the 6th NASA Conference on Evolvable Hardware. (2005) 87–92 17. Wagner, A.: Robustness against mutations in genetic networks of yeast. Nature Genetics 24 (2000) 355–361 18. Stanley, K., Miikulainen, R.: A taxonomy for artificial embryogeny. Artificial Life 9(2) (2003) 93–130 19. Kitano, H.: Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4:4 (1990) 461–476 20. Gruau, F.: Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon (1994) 21. Hornby, G., Pollack, J.: Body-brain co-evolution using L-systems as a generative encoding. In Spector, L., al., eds.: Proc. of the Genetic and Evolutionary Computation Conference, GECCO-2001, Morgan Kaufmann (2001) 868–875 22. Hornby, G., Pollack, J.: The advantages of generative grammatical encodings for physical design. In: Proc. of the 2001 Congress on Evolutionary Computation, CEC 2001, IEEE Press (2001) 600–607 23. Bongard, J.: Evolving modular genetic regulatory networks. In: Proc. of the 2002 Congress on Evolutionary Computation (CEC2002), IEEE Press, Piscataway, NJ, 2002 (2002) 1872–1877 24. Dellaert, F., Beer, R.: Toward an evolvable model of development for autonomous agent synthesis. In R.Brooks, Maes, P., eds.: Proc. of Artificial Life IV, MIT Press Cambridge (1994) 246–257 25. Eggenbergen-Hotz, P.: Evolving morphologies of simulated 3d organisms based on differential gene expression. In Husbands, P., Harvey, I., eds.: Proc. of the 4th European Conference on Artificial Life (ECAL97). (1997) 205–213 26. Cangelosi, A., Nolfi, S., Parisi, D.: Cell division and migration in a ’genotype’ for neural networks. Network: Computation in Neural Systems 5 (1994) 497–515 27. Ohno, S.: Evolution by Gene Duplication. Springer (1970) 28. Nowak, M.: What is a quasi-species? Trends Ecol. Evol. 7 (1992) 118–121