WHEN TO WALK AWAY AND WHEN TO STAY: COOPERATION EVOLVES WHEN AGENTS CAN LEAVE UNCOOPERATIVE PARTNERS AND GROUPS C. Athena Aktipis
A DISSERTATION in Psychology For the Graduate Group in Psychology
Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2008 _______________________________ Robert Kurzban, Dissertation Supervisor _______________________________ Jon Baron, Graduate Group Chairperson
COPYRIGHT Christina Athena Aktipis 2008
iii
DEDICATIONS To my Mother: This dissertation is dedicated to the memory of my mother, Helga Aktipis (2000). Her unconditional love, unwavering belief in me and her support of my educational goals have continued to inspire me and enable my work. Thank you, Mom, for making sure I had the tools I needed to carve my own path.
And to my children: May you ‘walk away’ from the more traveled path with courage, grace, and at least some idea of where you’re going.
iv
ACKNOWLEDGEMENTS The work culminating in this thesis began during my year as a course developer and independent scholar at Portland State University’s Systems Science Department. Thank you to Martin Zwick and Wayne Wakeland for enabling that position, supporting my agent based simulation work, and engaging with me on the topic of multilevel selection. Jeff Fletcher, who was a graduate student at Portland State at the time, was a lively colleague at the beginning of his career modeling multilevel selection. The early encouragement of Ed Hagen and Nicole Hess during a visit to Berlin prompted me to take my initial explorations into the Walk Away strategy more seriously. The interest and guidance of David Sloan Wilson during a visit to Binghamton helped me to find the most effective channels for presenting and publishing my work. Thank you to these early supporters and friends. To all the mentors I’ve had over the years, I cannot say enough. Thank you to Joy Joyce for an engaging high school introduction to experimental economics, Edward Raddatz for taking pride in me, John Mostacci for memorable in-class ‘lobotomies,’ Noelwah Netusil for saving me from potential disaster, Melissa Rutherford for tolerating what must have been at least mildly annoying constant questioning, Allen Neuringer for reminding me that self-experimentation is always an option, Peter Todd for entertaining possibilities, and to Sharon Thompson-Schill for restoring my faith in humanity. The members of the Kurzban lab group have been great sources of lively banter and intellectual stimulation. Thank you especially to Peter Descioli, Marc Egeth, and Alex Shaw for talking with me about my wild ideas and offering some creative ones in
v return. And of course, thank you to Rob Kurzban for invaluable feedback, comments and advice throughout my time at Penn. Thank you to all the members of my dissertation committee for committing the time and energy to help me improve my work. The support, guidance and encouragement of Robert Seyfarth and Frank Norman over the years have helped me to navigate dissertation process and keep my spirits up. Hans-Peter Kohler introduced me to new topics and areas within Sociology that will continue to influence my work. Jon Baron, during his short time on my committee, helped me to see that there are many different approaches to scientific inquiry. John Sabini passed away in 2005, shortly after becoming a member of my committee. His genuine interest in my intellectual development and encouragement of my work continue to inspire me, despite my sadness that he is no longer with us. Thank you to my parents and my brother, who have always supported my academic work. My late mother, Helga, not only inspired me, but showed me how find inspiration in the world. Thank you to my Dad, Stelios, for the reminders during grade school that I’d be much happier once I was in College and Graduate School. You were right. Thank you also to my brother Michael, whose immense success has been an alternating source of comfort and discomfort to me since we were young children. Remember, Michael, no matter what you do, I will always be older. Without the help of a dedicated crew of babysitters, family members and other helpers, this dissertation might not have seen the light of day for another several years. Thank you to Stephanie Collins, Mike Dowden, Dan Tague, Laura Sockol, Lacinda Benjamin, Francine Robinson, Dorene Warner and Dave and Rachel Castro-Diephouse
vi for helping our household run more smoothly during these dissertation years. Thank you also to Jon Alder at the Beauty Shop Café for providing various sources of caffeine that helped my brain run more smoothly during the final months and weeks of writing. My partner, Josh Warner, has played every role in supporting the completion of this dissertation: he has been a sounding board for ideas, a source of encouragement and advice, and a dedicated husband and father throughout. Thank you, Josh, for dropping Avanna off at school in the morning, feeding the cats, feeding the rest of us, putting Monty to sleep at night, doing the bills and putting away dishes - and then somehow finding the energy to talk with me about my dissertation until three in the morning. I truly could not have done it without you. Thank you also, to my children, Avanna and Monty, for reminding me that there is always time to smile, take off my socks, and dance.
vii
ABSTRACT
WHEN TO WALK AWAY AND WHEN TO STAY: COOPERATION EVOLVES WHEN AGENTS CAN LEAVE UNCOOPERATIVE PARTNERS AND GROUPS
C. Athena Aktipis
Dissertation Supervisor: Robert Kurzban
Cooperation among group members, coworkers and community members can provide benefits for all involved parties. However, groups of all kinds are plagued by free riders, or individuals who take advantage of cooperative group members by benefiting from being a part of the group without contributing, resulting in a social dilemma or 'tragedy of the commons.' This phenomenon is not unique to humans; free riders can be identified in organisms as simple as bacteria. This has lead to the puzzling question of how cooperation is maintained in social groups of humans and other animals, given higher payoffs for free riding than for cooperation. In order to address this question, I simulate individuals who use a simple Walk Away rule to leave uncooperative partners or groups, and show that cooperation is favored under a variety of parameter values when agents can use this rule. When agents use the Walk Away rule, more cooperative partnerships and
viii groups are more stable than less cooperative ones. This promotes assortment, or the preferential interaction of cooperators with one another, which favors the evolution of cooperation. It is shown that in dyadic partnerships Walk Away can outperform the wellknown Tit-for-Tat strategy. In group-wise interactions, the Walk Away rule generates large number of relatively small groups and differential group stability based on average cooperativeness. These features maintain selection for cooperation by generating population structures that promote group selection. The simple Walk Away rule does not require complex individual level abilities such as long-term memory, recognition of group members or punishment, suggesting that complex cognitive abilities are not necessary for cooperation to be promoted.
ix
TABLE OF CONTENTS CHAPTER 1: ASSORTMENT IN SPACE AND TIME
1
1.0. ABSTRACT
1
1.1. INTRODUCTION
2
1.2. ASSORTMENT: A COHESIVE CONCEPTUAL FRAMEWORK FOR MODELS OF COOPERATION 1.3. CONTINGENT BEHAVIOR LEADS TO ACTIVE ASSORTMENT
6 12
1.3.1. Contingent benefit transmission
13
1.3.2. Responsive movement
14
1.4. ASSORTMENT IN SPACE AND TIME
15
1.4.1. Spatial Assortment
16
1.4.2. Temporal Assortment
19
1.4.3. Information Processing Leads to Interactions between Spatial and Temporal Assortment
26
1.5. AGENT-ENVIRONMENT INTERACTION IN BEHAVIORAL ECOLOGY
29
1.6. A ‘SIMPLE’ MODEL OF ACTIVE ASSORTMENT
33
1.7. DISCUSSION
41
1.8. CONCLUSION
45
1.9. OVERVIEW OF REMAINING CHAPTERS
47
CHAPTER 2: KNOW WHEN TO WALK AWAY
48
2.0. ABSTRACT
48
2.1. INTRODUCTION
49
2.1.1. Simple strategies
x 50
2.1.2. Game-theoretic models
51
2.1.3. Spatiality and movement
52
2.1.4. Exit and contingent cooperation
53
2.2. STRATEGY DESCRIPTION
54
2.3. METHODS (SIMULATION DESCRIPTION)
56
2.4. RESULTS
58
2.4.1. Basic model (Simulation 1)
58
2.4.2. Comparative performance of TFT and PAVLOV (Simulations 2-4)
60
2.4.3. Invasion (Simulations 5-6)
62
2.4.3.1. Simulation 5: Resistance to invasion
62
2.4.3.2. Simulation 6: Invasion of cooperation
64
2.4.4. Error (Simulation 7)
65
2.4.5. Movement Cost (Simulation 8)
66
2.4.6. Density (Simulation 9)
68
2.4.7. Increased Mutual Defection Payoff (Simulation 10)
69
2.5. DISCUSSION
70
2.5.1. Contingent Movement and Assortment
70
2.5.2. Comparing Walk Away to TFT and PAVLOV
71
2.5.3. Similar Models of the Evolution of Cooperation
73
2.5.4. Simple Strategies Revisited
74
2.6. CONCLUSION
76
xi
CHAPTER 3: WALKING AWAY FROM THE HAYSTACK
78
3.0. ABSTRACT
78
3.1. INTRODUCTION
79
3.1.1. Walk Away
82
3.1.2. Social Dilemmas and Multilevel Selection
84
3.2. MODEL RESULTS
87
3.2.1. Cooperator Viability
88
3.2.2. Migration
91
3.2.2.1. Migration dynamics in the absence of selection
92
3.2.2.2. Migration dynamics with selection
94
3.2.3. Dynamic population structures
96
3.2.3.1. Dynamics in a single run
96
3.2.3.2. Group Stability
98
3.3. DISCUSSION
103
3.3.1. Group and Multilevel Selection
104
3.3.2. Individuals Rules Affect Population Dynamics
106
3.4. METHODS
109
CHAPTER 4: THE INDIVIDUAL IN GROUP SELECTION
111
4.0. ABSTRACT
111
4.1. INTRODUCTION
112
4.1.1. Walk Away
113
4.1.2. Background
114
4.1.3. Complexity and Simplicity of Decision Rules
xii 114
4.2. SEMI-ISOLATED POPULATIONS
115
4.3. RESPONSIVE MOVEMENT AND GENETIC GROUP SELECTION
117
4.3.1. Responsive Movement and Individual Level Selection
117
4.3.2. Responsive Movement, Assortment and Levels of Selection
117
4.4. A MODEL OF GROUP SELECTION AND ASSORTMENT 4.4.1. Agent-Based Models of Genetic Group Selection 4.5. DISCUSSION 4.5.1. Migration and Differential Stability of Groups
119 119 127 127
4.5.1.1. Differential extinction of groups
128
4.5.1.2. Permeability of group boundaries
129
4.5.2. Responsive Movement and Cooperation in Humans 4.5.2.1. Migration and instability in human groups
129 130
4.5.3. Interactions between Levels of Selection in the Evolution of Cooperation
133
4.5.4. If Genetic Group Selection is Plausible, Why is Cooperation Rare?
133
4.6. CONCLUSIONS
135
CHAPTER 5: CONTINGENT MOVEMENT AND BENEFIT TRANSMISSION IN THE EVOLUTION OF COGNITION AND SOCIAL BEHAVIOR
136
5.0. ABSTRACT
136
5.1. INTRODUCTION
137
5.2. CONTINGENT BEHAVIOR
138
5.2.1. Contingent Movement
xiii 141
5.2.1.1. Walk Away
142
5.2.1.2. Contingent Benefit/Cost Emission
143
5.2.1.2.1. Benefit Emission
143
5.2.1.2.2. Cost Emission
145
5.3. THE CALIBRATION OF THRESHOLDS FOR CONTINGENT BEHAVIOR
146
5.3.1. Calibration of Walk Away based on previous Temporal Assortment Information 5.3.2. Calibration of Benefit Emission based on Spatial Assortment Information 5.4. CALIBRATION OF OUTPUT
147 148 151
5.4.1. Calibration of movement
152
5.4.2. Calibration of benefit emission
153
5.5. THE VULNERABILITY OF INPUT CONDITIONS
155
5.5.1. From passive to active signals
159
5.5.2. Honest and deceptive signals
163
5.5.2.1. Exploiting contingent movement
165
5.5.2.2. Exploiting contingent benefit transmission
167
5.6. STRATEGIC INTERACTIONS AND CONTINGENT SIGNALING
170
5.6.1. Fixed Signals
172
5.6.2. Channel Contingent Signaling
172
5.6.3. Resource Availability Contingent Signaling
176
5.7. EMBEDDED CONTINGENT RULES 5.7.1. Embedded movement rule representation
178 178
5.7.2. Embedded benefit transmission rule representation
xiv 180
5.7.3. Embedded mental state representation
181
5.8. DISCUSSION
183
5.9. CONCLUSION
188
APPENDIX A
191
APPENDIX B
193
BIBLOGRAPHY
194
xv
LIST OF TABLES TABLE 1.1 TYPES OF ASSORTMENT. ...............................................................................18 TABLE 1.2. A FRAMEWORK FOR MODELS OF THE EVOLUTION OF COOPERATION .............25 TABLE 2.1. PERCENTAGE OF AGENTS OF EACH TYPE IN SIMULATIONS 1-5.......................63 TABLE 2.2. PERCENTAGE OF AGENTS AT VARIOUS ERROR RATES ...................................65
xvi
LIST OF FIGURES FIGURE 1.1 TYPES OF ASSORTMENT. ..............................................................................28 FIGURE 1.2 PASSIVE ASSORTMENT IN SIMPLE MODEL . ................................................36 FIGURE 1.3 ACTIVE ASSORTMENT IN SIMPLE MODEL. ..................................................40
FIGURE 2.1 PRISONER’S DILEMMA PAYOFF MATRIX.......................................................50 FIGURE 2.2 STATE TRANSITION FIGURES FOR WALK AWAY AND NAIVE..........................54 FIGURE 2.3 STATE TRANSITION FIGURES FOR TFT AND PAVLOV..................................56 FIGURE 2.4 NUMBER OF AGENTS OF EACH TYPE (SIM 1).................................................59 FIGURE 2.5 NUMBER OF AGENTS OF EACH TYPE (SIM 4).................................................61 FIGURE 2.6 COMPARASION OF INVADABILITY.................................................................62 FIGURE 2.7 NUMBER AGENTS AT VARIOUS DENSITIES....................................................67
FIGURE 3.1 STATE TRANSITION FIGURE FOR WALK AWAY. ............................................84 FIGURE 3.2 PERCENT COOPERATORS WITH VARIED THRESHOLDS. ..................................89 FIGURE 3.3 PERCENT COOPERATORS WITH VARIABILTY OF THRESHOLDS........................90 FIGURE 3.4 MIGRATION RATES IN THE ABSENCE OF SELECTION. .....................................94 FIGURE 3.5 MIGRATION RATES WITH SELECTION............................................................96 FIGURE 3.6 POPULATION DYNAMICS AND SELECTION FOR COOPERATORS. ......................97 FIGURE 3.7 WALK AWAY AT THE INDIVIDUAL, GROUP AND POPULATION LEVEL.............99
xvii FIGURE 3.8 POPULATION STABILITY AND SUBDIVISION. ................................................ 100 FIGURE 3.9 ANALYTICAL VS. AGENT-BASED MODELS OF GROUP SELECTION................ 102
FIGURE 4.1 STATE TRANSITION FIGURE FOR WALK AWAY ........................................... 122 FIGURE 4.2 SNAPSHOTS OF GROUP DYNAMICS. ............................................................ 125 FIGURE 4.3 SCHEMATIC OF SIMPSON’S PARADOX. ........................................................ 127
FIGURE 5.1 STATE TRANSITIONS FOR WALK AWAY AND BENEFIT EMISSION. ................ 145 FIGURE 5.2 CALIBRATION OF THRESHOLDS FOR CONTINGENT BEHAVIOR...................... 150 FIGURE 5.3 CALIBRATION OF OUTPUT OF CONTINGENT BEHAVIOR. .............................. 154 FIGURE 5.4 PASSIVE, POSITIVE AND NEGATIVE SIGNALS. ............................................. 161 FIGURE 5.5 HONEST AND DECEPTIVE TEMPORAL ASSORTMENT SIGNALS. ..................... 166 FIGURE 5.6 HONEST AND DECEPTIVE SPATIAL ASSORTMENT SIGNALS. ......................... 169 FIGURE 5.7 CHANNEL CONTINGENT SIGNALS. .............................................................. 174 FIGURE 5.8 ECOLOGICALLY CONTINGENT SIGNALS. ..................................................... 177 FIGURE 5.9 EMBEDDED CONTINGENT RULES. ............................................................... 180 FIGURE 5.10 MENTAL STATE AND SOCIAL CONTINGENT RULE REPRESENTATIONS. ....... 182 FIGURE 5.11 CULTURAL TRANSMISSION . ..................................................................... 187
1
CHAPTER 1 ASSORTMENT IN SPACE AND TIME: A FRAMEWORK FOR THE EVOLUTION OF COOPERATION AND RESPONSIVE MOVEMENT
1.0. ABSTRACT Despite a proliferation of theories and models of the evolution of cooperation, there is little consensus about the mechanisms that are most important to the selection and maintenance of cooperative behavior. The seeming paradox of the evolution of cooperative behavior is less puzzling when a focus is placed on the role of assortment. The major goals of this chapter are to 1) describe a novel framework for understanding the evolution of cooperation based on assortment in space and time, introducing the notion of temporal assortment 2) articulate the general importance of distinguishing passive and active assortment in the evolution of cooperation, and 3) describe why responsive movement in particular is likely to have played an central role in the evolution of cooperation. This chapter proposes a 2x2 framework distinguishing passive/active assortment and temporal/spatial assortment of gene copies, providing a framework for comparing existing models of the evolution of cooperation. Further, this approach suggests connections between areas of research in biology and fruitful new directions for research on the evolution of social behavior. The analytical intractability of modeling active assortment and subtle interactions favors agent based simulations for a variety of
2 questions about the evolution of cooperation and movement. This chapter introduces an agent-based simulation, SIMPLE (Simple agent Interaction through Movement and Production in a Local Environment), and uses this model to demonstrate several fundamental concepts in the evolution of movement and benefit transmission.
1.1. INTRODUCTION Traditionally, debates about the mechanisms underlying the evolution of cooperation have focused on the relative importance of kin-selection, multilevel selection, reciprocity and complex cognitive/behavioral strategies. Considering the controversy that tends to surround theories about the evolution of cooperation, it is relatively surprising that assortment plays a central role in the operation of a wide variety of these proposed mechanisms. Assortment is a way of conceptualizing the proximity of individuals (or genes) of the same type within groups of interacting individuals. Positive assortment, or simply assortment, is the aggregation of the same type within groups, while negative assortment is characterized by less similarity within groups than in the general population. Kin selection (e.g., Hamilton 1964a; b), group or multilevel selection (Maynard Smith 1964; Price 1970; Wilson 1987), reciprocity (Axelrod 1984; Axelrod & Hamilton 1981; Trivers 1971) and selection in structured populations such as those on lattices (e.g., Brauchli et al 1999; Ifti et al 2004; Nakamaru 2006; Nowak & May 1992) or graphs (e.g., Lehmann et al 2007; Ohtsuki et al 2006; Taylor et al 2007) all operate on assortment. This section provides an overview of such theories of the evolution of cooperation, noting that assortment is a fundamentally important aspect of the evolution of cooperation
3 within these divergent approaches. Models of the evolution of cooperation draw on the formal tools of game theory (von Neumann & Morgenstern 1944), typically making use of social dilemma (Dawes 1980) paradigms. Social dilemmas are characterized by a conflict between the individually optimal decision (to defect) and the collectively optimal decision (to cooperate). Because of this conflict between the individually and collectively optimal behaviors, social dilemmas are excellent formal tools for exploring the evolution of social behavior, capturing the tension in benefit optimization at different levels of organization. In dyadic models of cooperation, the prisoner’s dilemma and the hawks/doves (also called the snowdrift game) are common (see Doebeli et al 2005 for a review). Group based interactions are often modeled after public goods games (Ledyard 1995) and the ‘tragedy of the commons’ (Hardin 1968). Explaining the evolution of cooperative behavior has been a long-term goal in theoretical evolutionary biology. Early field work on the evolution of cooperation focused on behaviors thought to be group and species level adaptations (Wynne-Edwards 1962; Wynne-Edwards 1963). Formals models of the evolution of cooperation soon challenged this work, and the individual and gene based views which focused on the average fitness of genes (Dawkins 1976/1989; Maynard Smith & Price 1973; MaynardSmith 1964; Williams 1966) came to be more widespread and accepted than ‘good of the group’ approaches. A major component of this was the formalization of the idea of an evolutionarily stable strategy (ESS), a strategy that cannot be invaded by a mutant strategy when individuals randomly encounter others in an infinite population (Maynard Smith & Price 1973).
4 It was, however, recognized that interactions are often not random in the natural world because of limited dispersal of offspring and local interactions, which give rise to interactions with kin. This is captured in inclusive fitness and kin selection models, where individuals preferentially assort with and cooperate with kin, allowing for cooperation to evolve (Hamilton 1964a; b; Maynard Smith 1964). The fundamental mechanism underlying the operation of kin selection is the selection of genes for a trait (cooperativeness) because of the positive effects the trait has on individuals that share the gene through common descent. The well known equation describing this effect is called “Hamilton’s Rule,” c < b*r; it specifies the cost (c) to benefit (b) ratio required for altruism to evolve between individuals of varying degrees of relatedness (r). Multilevel/group selection has had a more controversial history (see Sober & Wilson 1998; Wilson 1983; Wilson & Wilson 2007). However, the fundamental features of multilevel selection are the same as those of kin selection: they involve the selection of genes for cooperative traits because of positive effects on others that share (genes that code for) those traits (Maynard Smith 1964; Price 1970; Wilson 1987; Wilson & Wilson 2007). The mathematics underlying group selection are measuring the within vs. between group variance in a trait through the variance ratio or another index of assortment (e.g., Pepper 2000). When there is variation between groups in cooperativeness of the members, the groups with a higher proportion of cooperators are more successful. It has been shown that kin selection can be conceptualized as special case of multilevel/group selection where ‘groups’ are made up of interacting kin (Queller 1992).
5 The role of reciprocity in the evolution of cooperation has been the focus of much theoretical and empirical work (Axelrod 1984; Axelrod & Hamilton 1981; Bowles & Gintis 2004; Boyd & Richerson 1988; Fehr et al 2002; Gintis 2000; Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003; Panchanathan & Boyd 2004; Trivers 1971). Reciprocity promotes cooperation through long-term benefits to the individual from mutually cooperative repeated interactions. These cooperative interactions are possible without great risk to either partner if they each follow a reciprocal Tit-for-Tat strategy that copies the behavior of their partner on the last round (Axelrod 1984; Axelrod & Hamilton 1981). A Tit-for-Tat strategy allows individuals to cooperate when interacting with cooperators and to switch to defection if they encounter defection, protecting Tit-for-Tat from exploitation, while simultaneously allowing for benefits from repeated interactions with cooperators. This ability to use past behavior as a signal of likely future behavior results in higher long-term payoffs for cooperator partners compared to defector partners. A variety of more complex and information intensive strategies have been implicated in the evolution of cooperation. For example, relatively recent work has investigated the role of memory (Aktipis 2006; Cox et al 1999; deVos & Zeggelink 1994; Mealy 1996; Milinski & Wedekind 1998), reputation (Milinski et al 2002), gossip (Nakamaru & Kawata 2002), indirect reciprocity (Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003; Panchanathan & Boyd 2004), commitment (Nesse 2001), norm following (Fehr & Fischbacher 2004; Fehr et al 2002), policing/punishment (Brandt et al 2006; Fowler 2005; Gardner & Westt 2004) and tag-based cooperation (Hammond & Axelrod 2006; Spector & Klein 2006) as possible mechanisms enabling the evolution
6 of cooperation, particularly in humans. Like reciprocity, each of these strategies increases the viability of cooperation through some form of signal processing that decreases uncertainty about a potential partner’s behavioral characteristics and a facultative response to that information. Another area that has received attention is that of partner choice, or exit from interaction. Partner choice has been shown to be a potentially important influence on cooperation in humans (Barclay & Willer 2007; Boone & Macy 1999; Orbell et al 1984; Schuessler 1989) and various models of partner choice have demonstrated the important role that partner choice can play in the evolution of cooperation (Aktipis 2004; Ashlock et al 1996; Connor 1992; Cox et al 1999; Enquist & Leimar 1993; Eshel & Cavalli-Sforza 1982; Hamilton & Taborsky 2005; Noe & Hammerstein 1994; Vanberg & Congleton 1992). As a result of this individual level partner choice behavior, partnerships and groups with more cooperators can end up being more stable and producing more offspring, leading to greater success for cooperators.
1.2. ASSORTMENT: A COHESIVE CONCEPTUAL FRAMEWORK FOR MODELS OF COOPERATION Each of the approaches described above has contributed importantly to our understanding of the evolution of cooperation. However, confusion arises when kin selection, group selection, reciprocity, byproducts and direct benefits are conceptualized as alternative categories in which to place instantiations of the evolution of cooperation. These different approaches rely on the same fundamental principle: that genes having
7 positive effects on copies of themselves will be selected. When individuals with shared genes preferentially assort with one another, this can promote cooperation. In fact, the inclusive fitness framework was developed as a more general approach focusing on positive regression of genotype rather than kinship per se (Hamilton 1975). An underappreciated source of positive regression of genotype (assortment) is that which can arise from the systematic interaction of individuals of different types with their local environments (Hamilton 1975; Pepper & Smuts 1999; Pepper & Smuts 2002; Wilson 1977). This general principle (that genes positively affecting copies of themselves will be selected) is difficult to ascertain when looking at recently proposed frameworks for categorizing the evolution of cooperation. In the past several years a number of papers have arisen that treat different approaches to conceptualizing assortment as categories for distinguishing between types of cooperation (e.g., Lehmann & Keller 2006; Nowak 2006; Sachs et al 2004). One of the attractions of such an approach might be that it categorizes types of cooperation based on units that are intuitively delineable, such as kin (in kin selection), group members (in group selection), long-term partners (in reciprocity), nearby individuals (in byproducts) and the future self (in direct benefits). However, the intuitive appeal of such classification systems seems to come at the expense of weak underlying conceptual structure. Proposed classification systems have included four or more primary categories (Lehmann & Keller 2006; Nowak 2006), or seven secondary categories (Sachs et al 2004) that bear little or no systematic relation with other categories within the same classification system. An attempt to create a systematic framework that does not treat different approaches to assortment as categories has
8 proposed four binary questions that result in sixteen distinct types of cooperation (Bergmuller et al 2007a). There is a clear need for a succinct and well structured framework in which to understand the evolution of cooperation. Another difficulty with existing frameworks for the evolution of cooperation is that they often collapse evolutionary outcomes with individual level outcomes and individual behavior. Kin selection and group/multilevel selection are evolutionary processes that can change the frequency of cooperative genes in the population given the proper spatial and temporal structure. Direct benefits, indirect benefits and byproduct benefits are ways of describing the individual level outcomes of cooperative interactions between and among individuals. Reciprocity and other strategies involving information processing are instantiations of individual level decision rules. Existing classification systems collapse these aspects, making more opaque the distinctions among mechanisms that promote selection for cooperation (those relying on the spatial and temporal structure of the population), the mechanisms that instantiate cooperation (the transmission of benefits) and mechanisms that enable conditional behavior (such as reciprocity). A classification system that respects these distinctions would be a valuable conceptual tool. The framework proposed in this chapter preserves distinctions between evolutionary and behavioral mechanisms within a succinct classification system based on types of assortment. Assortment is a natural choice for a unifying principle around which to understand the evolution cooperation. Mechanisms that instantiate cooperation often act on assortment or promote assortment, and there is little disagreement about the importance of spatial assortment in the evolution of cooperation. Previous work has suggested that an important distinction can be drawn between passive assortment, which
9 operates on structural features of the environment, and active assortment, which involves decision making on the part of the organism (Eshel & Cavalli-Sforza 1982; Taylor & Day 2004). This distinction between passive and active assortment is central to the classification system presented here. The present system also introduces the concept of temporal assortment, which, along with spatial assortment, completes a framework for understanding the evolution of cooperation and other behaviors that is concise, straightforward and suggests important directions for new work. Temporal assortment is a way of conceptualizing the effect that genes can have on future copies of themselves, including, but not limited to, effects that on the same individual in future time periods. The strength of temporal assortment and the resulting fitness effects will be based on the nature of the coupling (long-term association) of the individual with its local environment. For example, consider a gene that promotes exploitative behavior. Over time, the local environment will become depleted. If the individual is unable to change its local environment, this will have long-term negative effects on copies of the gene. On the other hand, a gene coding for productive behavior will positively affect copies of itself in future time periods as long as the coupling with the environment is maintained. The importance of long-term social interactions has long been considered in approaches to the evolution of cooperation (Axelrod 1984; Axelrod & Hamilton 1981; Bull & Rice 1991; Sachs et al 2004; Trivers 1971). However, the present approach considers temporal assortment as future effects on genes resulting from interaction with both the social and physical environment. Coupling (positive temporal assortment) and decoupling (negative temporal assortment) with the local environment can greatly affect future payoffs, with extended
10 coupling being advantageous for producers and decoupling being advantageous for exploiters. The ability of individuals to change their local environments by moving in space is likely to have important evolutionary implications. Further, the ability to use information about present (and estimated future) payoffs to guide movement rules allows for flexible and adaptive coupling/decoupling with the local environment, as in optimal foraging models (MacArthur & Pianka 1966; Stephens & Krebs 1986). Analytical models of the evolution of cooperation typically do not include the possibility for movement in space and agent based models provide a distinct advantage for modeling contingent movement because they allow for the combination of individual characteristics with spatial structure (Hammond & Axelrod 2006). This allows for individual level interaction with the physical (Pepper & Smuts 1999; Pepper & Smuts 2002) and social (Aktipis 2004) environment to affect the spatial association of individuals. When individuals use contingent movement to respond to and optimize the payoffs from extended coupling with the environment (temporal assortment), this can effectively create spatial assortment. However, most analytical models take behavioral assortment (or its precursors) as a variable to change parametrically, making it difficult to investigate the role of active assortment in the evolution of cooperation. Classic models of kin selection compute whether an altruistic behavior will be selected using degree of kinship as an independent variable (Hamilton 1964a; b) and multilevel selection models typically rely on the ratio of between-group variance to within-group variance, which is not assumed to change due to active choice on the part of the individuals within the group (Maynard Smith 1964). Kinship and the ratio of between to within-group variance are
11 ways of describing spatial assortment. The nature of analytical models makes it easy to vary initial assortment, but more difficult to investigate the role of active behavioral assortment. Because of the difficulties in creating analytical models that capture the interaction between active behavioral assortment and spatial dynamics, the importance of these processes in promoting the evolution of cooperation may have been underestimated in work to date. The framework outlined here describes several pathways to assortment that promote evolution of cooperation, proposing a 2x2 framework that distinguishes passive from active assortment and temporal assortment from spatial assortment of genes. This classification system does not emphasize features that have been traditionally seen as central differences in types of altruism, such as the distinction between the provision of benefits to the self, benefits to kin and benefits to others that share genes for reasons other than decent (Lehmann & Keller 2006; Nowak 2006; Sachs et al 2004). Instead, this classification system focuses on the spatial and temporal association of genes with copies of themselves via direct interactions or indirect interactions through the local environment. Because genes can influence the viability of copies of themselves that are distant in space and/or time, selection would be expected to optimize net benefits to genes over both spatial and temporal tradeoffs. Other proposed frameworks for the evolution of cooperation (Bergmuller et al 2007a; Lehmann & Keller 2006; Nowak 2006; Sachs et al 2004) preserve the distinction between effects on others from effects on the self, because it is thought that behaving in a way that benefits the self in the future does not constitute an altruistic or cooperative act. The present classification system does not make any categorical distinction based on the
12 entities that are the recipients of the benefits. Instead, cooperation is conceptualized as any benefit providing act in which there is statistical uncertainty about the benefits and/or costs accruing to copies of the gene in at various spatial and temporal distances. This draws on certain aspects Buston and Balshine (2007) uncertainty approach to conceptualizing acts of cooperation. Because this is an untraditional use of the term ‘cooperation,’ the terms ‘benefit emission’ and ‘benefit transmission’ are favored throughout this thesis.
1.3. CONTINGENT BEHAVIOR LEADS TO ACTIVE ASSORTMENT The ability to respond to information in the environment in ways that promote the long terms success of genes is highly adaptive. Most organisms take in information from the environment and respond contingently, either through changes in gene expression in response to environmental cues or through neural systems that process incoming perceptual inputs. The ability to respond contingently enables individuals to engage in behavior that promotes their own survival and reproduction. Furthermore, contingent behavior creates effects on the social and physical environment that can have important influences on the proximity and viability of copies of genes that are near in space and time. In other words, the ability to respond to information from the environment can lead to changes in assortment. The processing of information from the environment is the essence of active assortment, whether from the social environment (Eshel & CavalliSforza 1982; Pepper 2007; Taylor & Day 2004) or the physical environment (Pepper & Smuts 1999; Pepper & Smuts 2002).
13 It is proposed that active assortment is guided by two fundamental types of contingent behavior, 1) the conditional transmission of benefits (or costs) and, 2) responsive movement or other restructuring of the local environment. Both types of contingent behavior take information from the environment and operate on the information with adaptive decision rules. The fit between decision rules and the beneficial effects resulting from those rules determines whether or not the genes underlying that decision rule will be selected by evolution. Various contingent benefit transmission rules have been explored in past work, but few models have explored the adaptive value of contingent movement rules. A brief overview of these two types of contingent behaviors is provided below. 1.3.1. Contingent benefit transmission Many models of the evolution of cooperation have explicitly or implicitly addressed the role of contingent benefit transmission. Kin selection (Hamilton 1964a; b) can favor the evolution of recognition mechanisms which allow for conditional transmission of benefits towards kin relative to non-kin. Reciprocity is based on the principle conditional cooperation (Trivers 1971), and the famous reciprocal Tit-for-Tat strategy (Axelrod 1984; Axelrod & Hamilton 1981) is successful because it transmits benefits only to others with a history of past cooperation, i.e., other Tit-for-Tat agents or unconditional cooperators. The PAVLOV strategy (Nowak & Sigmund 1993) contingently transmits benefits only to others that contingently transmit benefits. Interestingly, PAVLOV occasionally ‘tests’ its partner by defecting and resumes cooperation only if the partner responded to defection by defecting. This allows
14 PAVLOV to reap the benefits of exploiting agents that do not contingently transmit benefits. Models of more complex information processing strategies such as those based on reputation, gossip or indirect reciprocity (as reviewed above) involve the contingent transmission of benefits. In these models, individuals transmit benefits only when the partner’s known interaction history is sufficiently cooperative. Contingent benefit transmission has been widely explored in models of the evolution of cooperation (see Table 2). However, few models of the evolution of cooperation have explicitly explored the role of movement in space. Despite greater attention to the contingent transmission of benefits, contingent movement in space might be as important of a fundamental principle guiding the evolution of cooperation. Contingent movement allows not only for individual level adaptive responses (leaving areas with insufficient benefits) but also changes in aggregate dynamics that can influence selection dynamics, ideas that are discussed in greater depth in future sections. 1.3.2. Responsive movement Responsive movement in space has been modeled in only a few published simulations (Aktipis 2004; Hamilton & Taborsky 2005; Pepper 2007; Pepper & Smuts 1999; Pepper & Smuts 2002), despite the apparent importance of spatial proximity in social interactions for most species. However, partner choice models capture the notion of ‘leaving’ uncooperative partners (Ashlock et al 1996; Connor 1992; Cox et al 1999; Enquist & Leimar 1993; Eshel & Cavalli-Sforza 1982; Noe & Hammerstein 1994; Vanberg & Congleton 1992) or restructuring the social network so that future interactions
15 to not occur with an uncooperative past partner (Fu et al 2007; Masuda & Aihara 2003; Santos et al 2006). When space is modeled explicitly and the behavior of individuals can have effects on the social and physical local environment, responsive movement can create important spatial dynamics that cannot emerge from other methods of partner choice such as refusing to play. In particular, responsive movement can create active spatial assortment, which can have important effects on the viability of the evolution of cooperation.
1.4. ASSORTMENT IN SPACE AND TIME In this section I discuss several different types of assortment and propose a classification system for the mechanisms that promote assortment. This framework focuses on two fundamental distinctions between types of assortment: passive vs. active assortment and spatial vs. temporal assortment. Passive assortment emerges as a byproduct of ecological and demographic processes. Active assortment, on the other hand, arises from the processing of information in the environment and contingent responses to information. Further, a distinction is made between processes that rely on temporal assortment of genes and those that rely on spatial assortment of genes. When a gene codes for behaviors that have effects on copies of the gene that are distant in time or space, this can affect selection pressures for cooperative behavior. The framework proposed here is both concise and has an underlying conceptual structure (Table 1.1), providing an alternative to classification systems that propose a moderate number of conceptually unrelated categories (Lehmann & Keller 2006; Nowak 2006; Sachs et al 2004), or a very large number of related categories (Bergmuller et al
16 2007a). This framework provides a straightforward way to classify the diverse work on the evolution of cooperation (Table 1.2), showing that a large number of models of the evolution of cooperation can be conceptualized in terms of spatial/temporal and active/passive assortment. The final row of Table 1.1 lists terms from the literature that have been used to describe phenomena falling into these categories. Another problem with previous frameworks of the evolution of cooperation is the collapse of evolutionary outcomes (selection, e.g., ‘kin selection’) individual level effects (benefits, e.g., ‘direct benefits’), and contingent rules (information processing, e.g., ‘reciprocity’). The present approach (summarized in Table 1.1) describes the individual level benefits/costs, evolutionary outcomes and (for active assortment) information processing components of each type of assortment. 1.4.1. Spatial Assortment Passive spatial assortment does not involve the processing of information, resulting simply from structural and demographic features of the population such as local reproduction with limited dispersal (Eshel & Cavalli-Sforza 1982; Taylor & Day 2004). When demographic features of the population generate spatial proximity of individuals that share genes, this is passive spatial assortment. In contrast, active spatial assortment is at work when individuals process information from the environment by moving or otherwise restructuring the population in a way that tends to lead to non-random spatial proximity of individuals that share genes. Passive spatial assortment is known to play an important role in models of kin selection (Hamilton 1964a; b), and multilevel selection (Maynard Smith 1964; Price
17 1970; Wilson 1987). Previous classification systems of cooperative interactions have delineated categories such as ‘kin fidelity’ (Sachs et al 2004), ‘kin selection’ (Lehmann & Keller 2006; Nowak 2006), and ‘indirect benefits’ (Bergmuller et al 2007a), which are TABLE 1.1 Types of Assortment Passive: Byproducts of ecological/demographic processes Passive Temporal
Cost/benefit structure
Information input
Future costs/benefits on genes from coupling with environment N/A
Passive Spatial Costs/benefits on genes from spatial proximity with copies
N/A
Active: Conditional response to social/ecological signals Active Temporal: Active Spatial: Contingent Benefit Contingent Transmission Movement Future costs/benefits on genes from coupling with environment
Costs/benefits on genes from spatial proximity with copies
Signals of costs/benefits from spatial assortment
Signals of costs/benefits from temporal assortment
Local benefit production, local resource consumption
Local reproduction, limited dispersal
Kin recognition, tag-based altruism, TFT, reputation/gossip/ indirect reciprocity
Benefit approach, Walk Away, optimal foraging
Social/physical environmental result
Fitness coupled with local physical or social environment
Proximity of kin that share genes
Change in cooperativeness or propensity to transmit benefits
Movement and/or restructuring of physical or social environment
Evolutionary result
Selection for genes that optimally trade-off costs and benefits to all copies of genes in various time periods
Selection for genes that optimally trade-off costs and benefits to all copies genes in nearby individuals
Selection for ability to contingently cooperate to optimize costs and benefits over all copies of genes in different spatial locations
Selection for ability to contingently move or restructure environment to optimize costs and benefits over time
Terms in the literature
Future benefits, structural assortment, partner fidelity feedback, environmental feedback
Indirect benefits, inclusive fitness, non-random matching, kin selection, structural assortment
Reciprocity, partner fidelity feedback, kin recognition, Titfor-Tat
Partner choice, exit, selective assortment, Walk Away
Instantiations
18 Table 1.1. The present classification system distinguishes between passive and active forms of assortment as well as temporal or spatial assortment of gene copies. Basic instantiations of these various types, as well as a description of potential environmental and evolutionary results are provided. The final column notes terms that have been used in the literature to describe mechanisms that fall into these categories, with some overlap of terms.
ways of describing the outcomes of spatial assortment on individuals and evolutionary processes. The present classification system focuses on the structural features of the population rather than the outcomes of interactions. This is in contrast to other classification systems which collapse the structural features with the individual level (i.e., indirect benefits) or evolutionary outcomes (i.e., kin selection) of preferential benefit transmission to kin. Despite the fact that preferential interactions with kin have been long considered an important factor in the evolution of cooperation, the importance of active spatial assortment in promoting the evolution has been underappreciated due largely to constraints of analytical models. Partner choice (Aktipis 2004; Ashlock et al 1996; Connor 1992; Cox et al 1999; Enquist & Leimar 1993; Eshel & Cavalli-Sforza 1982; Hamilton & Taborsky 2005; Noe & Hammerstein 1994; Vanberg & Congleton 1992) and network models (Fu et al 2007; Masuda & Aihara 2003; Santos et al 2006) have begun to address the importance of the active processing of information in promoting preferential interactions. However, many of these models do not explicitly embed individuals in a spatial environment leading to the loss of a potentially rich source of embedded ecological information (i.e., the statistical association between spatial proximity and genetic similarity) that can emerge from responsive movement. Agent based models that
19 explicitly model agents in space have demonstrated that when individuals are able to respond to information in the environment in ways that lead to spatial proximity and therefore preferential interactions with those that are likely to share genes, cooperation can be promoted (Aktipis 2004; Pepper & Smuts 1999; Pepper & Smuts 2002). The ability of individuals to use information embedded in the environment to make adaptive decisions has been called ‘ecological rationality’ (Gigerenzer et al 1999), a term that might apply widely to the decision making rules that operate on and effect assortment.
1.4.2. Temporal Assortment A novel aspect of the classification system proposed in the present chapter is the introduction of the notion of temporal assortment. Temporal assortment is analogous to spatial assortment. In spatial assortment, spatial proximity is considered keeping time constant (i.e., at a particular point in time). Analogously, temporal assortment can be considered as temporal proximity keeping space constant (i.e., for a particular local environment). In other words, spatial assortment involves copies of genes at different points in space while temporal assortment involves copies of genes at different points in time including (but not limited to) future copies of the self. To better cultivate intuitions about the nature of temporal assortment, consider a one-dimensional horizontal world that progresses down as time goes on, leaving a trail of past agent locations (for more details, see the section on the SIMPLE model). Spatial assortment can be conceptualized as the nonrandom spatial proximity with entities that share genes (represented in the SIMPLE model below by horizontal proximity). Similarly, temporal assortment can be
20 understood as the nonrandom temporal proximity of entities that share genes (represented in the model below by vertical proximity). The most straightforward form of temporal assortment of genes is the effect that individuals can have on their own payoffs in future time periods as a result of a long-term association with their physical and social environments. In other words, individuals that invest in their environment are likely to get benefits from that environment in the future. To the extent that highly investing individuals remain coupled with that environment, they can reap gains from the behavior of their past selves or the past behavior of other individuals (such as parents, other kin or social partners) that invested in or exploited the same local environment. Environmental feedback allows for payoffs of an individual to be influenced by the past behavior of itself and others in that local environment. This is an important way in which assortment can affect individual payoffs and subsequently the evolution of cooperation – proximity in time. Passive temporal assortment is the result of individual interactions with the environment in which they do not engage in information processing. In contrast, the ability to process information from the environment can lead to active temporal assortment, where the long term effects of an individual’s coupling with the local environment (on copies of the individual’s genes) are influenced by the contingent behavior. The simplest form of active temporal assortment is a change in exploitation of or investment in the local environment that affects future payoffs. In other words, a change in the emission of costs or benefits in the local environment can change the longterm effects of coupling with that environment. This might involve changes in the propensity to consume resources in the local environment or changes in the propensity to
21 cooperate or transmit benefits to interaction partners. For example, when agents interact with Tit-for-Tat players, the long-term beneficial effects of coupling with that partner can make the optimal strategy cooperation rather than defection, given a long enough future time frame and low discounting of future outcomes (Axelrod 1984). The idea of temporal assortment bears similarity to the notions of extended associations and partner fidelity feedback that have been discussed in earlier work (Axelrod 1984; Axelrod & Hamilton 1981; Bull & Rice 1991; Sachs et al 2004; Trivers 1971). However, in the present framework temporal assortment is not conceptualized as necessitating a specific interaction partner. Other classification systems simply conceptualize long-term benefits as ‘direct benefits’ (Lehmann & Keller 2006) or ‘direct reciprocity’ (Nowak 2006) from other social actors. Again, the temporal assortment category differs in that it captures both interactions with the social environment and interactions with the physical world that can have long-term effects on payoffs for copies of genes. At the most basic level, temporal assortment is the coupling of individuals with their local environment. If individuals exploit their local environments, this can result in long-term cost and if individuals invest in their environments, this can result in long-term benefits. The long term effects on individual investment in a partner has been proposed as an essential factor in the classification of the cooperative interactions in another recent framework (Bergmuller et al 2007b), but again the present framework includes interactions with the local environment more generally rather than with specific interaction partner.
22 TABLE 1.2 A Framework for Models of the Evolution of Cooperation Passive Temporal (Hamilton 1964a; b) (Maynard Smith 1964)
Metapopulation
(Trivers 1971)
Repeated interactions Repeated interactions Individuals are more likely to encounter same strategy
(Axelrod & Hamilton 1981) (Eshel & Cavalli-Sforza 1982)
(Peck & Feldman 1986)
(Wilson 1987)
Repeated cooperative interactions in dyads Metapopulations
(Dugatkin & Wilson 1992)
Metapopulation
(Killingback & Doebeli 1996; Nowak & May 1992)
Lattice, neighborhood interactions
(Vanberg & Congleton 1992)
Repeated PD encounters
(Enquist & Leimar 1993)
Repeated cooperative interactions in dyads or groups Repeated encounters of PD partners
(Nowak & Sigmund 1993)
Passive Spatial
Active Temporal: Contingent Benefit Transmission
Active Spatial: Contingent Movement
Interactions with kin Offspring placed in group, interactions within group, occasional migration Reciprocity Tit-for-Tat Individuals actively choose partners of same type
Offspring placed in group, occasional migration Defectors condition interaction on marginal value theorem Successful strategies repopulate neighbors
Offspring placed in group
Cooperation offered after cooperation is demonstrated Cooperation contingent on partner’s previous choice
Individuals refuse to play with known defectors Interaction propensity contingent on reputation
23 Passive Temporal (Lindgren & Nordahlb 1994)
Lattice, neighborhood interactions
(Ferriere & Michod 1995)
One-dimensional interaction space, local interactions
(Ferriere & Michod 1996)
One-dimensional interaction space, local interactions Repeated PD interactions
(Ashlock et al 1996)
(Epstein 1998)
Lattice, local interactions
(Brauchli et al 1999)
Lattice
(Cox et al 1999)
Repeated encounters of PD partners
(Mitteldorf & Wilson 2000)
Lattice, local interactions, some instability of neighborhood Metapopulation, local interactions
(Avilés 2002)
(Pepper & Smuts 2002)
Lattice, local interactions through shared use of renewable resource patches
Passive Spatial Successful strategies repopulate neighbors Offspring placed nearby (but with diffusion approximated mobility) Offspring placed nearby
Active Temporal: Contingent Benefit Transmission
Active Spatial: Contingent Movement
Cooperation contingent on partner’s previous behavior
Individuals refuse to interact with insufficiently cooperative partners Offspring placed on neighboring sites Cooperation contingent on partner’s previous behavior Cooperation contingent on partner’s previous behavior
Entering interaction contingent on partner’s previous behavior
Offspring placed on neighboring sites Linkage disequilibrium between group joining and cooperation Individuals leave areas with low resource levels
24 Passive Temporal (Marshall & Rowe 2003) (Suzuki & Arita 2003)
(Aktipis 2004)
(Fletcher & Zwick 2004)
Lattice, varied viscosity, local PD interactions One dimensional ring, N-player PD with neighbors Lattice, local interactions
(Gardner & Westt 2004)
Metapopulations, public goods interactions Metapopulation
(Ifti et al 2004)
Lattice, Network
(Hamilton & Taborsky 2005)
PD interactions among group members
(Fu et al 2007; Masuda & Aihara 2003; Santos et al 2006)
Network
(Ohtsuki et al 2006; Pacheco & Santos 2005; Santos & Pacheco 2006; Santos et al 2006; Santos et al 2005) (Zhang et al 2005)
Network
(Hammond & Axelrod 2006) (Hauert et al 2006),
Lattice, local PD interactions Local interactions High average payoff (high % cooperators) increases the growth rate
Passive Spatial Local reproduction
Active Temporal: Contingent Benefit Transmission Memory and Tit-for-Tat
Active Spatial: Contingent Movement
Limited dispersal Individuals move after encountering defectors
Facultative increase in punishment with higher local cooperativeness Successful strategies repopulation neighbors Interactions within groups but offspring disperse
Individuals condition their cooperation on past outcomes
Successful strategies repopulate neighbors Successful strategies repopulate neighbors Colonization of nearby patches Reproduction is local
Behavior is conditional on partner’s tag
Individuals leave group conditional on their own strategy and PD outcome Agents rewire network connections
25 Passive Temporal (Killingback et al 2006) (Spector & Klein 2006)
(Janssen & Goldstone 2006) (Traulsen & Nowak 2006)
Metapopulations One dimensional ring, interaction in varied neighborhood sizes Metapopulation, local public good interactions Metapopulation, within group altruistic interactions
(Fletcher & Zwick 2007)
Metapopulations, public goods interactions
(Ohtsuki & Nowak 2007)
Network
(Taylor et al 2007)
Network
(Wakano 2007)
Lattice
Passive Spatial
Local reproduction, limited dispersal
Active Temporal: Contingent Benefit Transmission
Active Spatial: Contingent Movement
Altruism performed only when tags are similar enough
Local reproduction Local reproduction
Regrouping occurs at temporal point when proportion of cooperators begins to decrease1* Direct reciprocity Interacting partners can displace one another upon reproduction Colonization of nearby patches, spatial diffusion of benefits
Table 1.2. This table provides a summary of the types of assortment at work in various agent-based, spatial and/or historically important models of the evolution of cooperation. Cooperation is promoted through passive temporal, passive spatial, active temporal and/or active spatial assortment.
1
In Fletcher and Zwick (2007), the time of regrouping is determined by the decrease in cooperators. In their simulations, agents did not actively process this information, but the timing of the regrouping was determined by the global procedures making use of a contingent rule that took in information from the environment and responded to that information by creating a change in grouping.
26 1.4.3. Information Processing Leads to Interactions between Spatial and Temporal Assortment Interactions between temporal and spatial assortment can emerge from active assortment. The ability of organisms to respond contingently to information in the environment can lead to changes in the structure of that environment and changes in the emission of benefits. This can lead to complex feedback loops wherein the active processing of information about temporal assortment leads to changes in spatial assortment and vice versa. A result of passive temporal assortment is a change in the long-term payoffs to individuals based on interactions with the local environment. An implicit assumption of the present approach is that entities affect one another’s fitness is through interactions with the environment. To the extent that genes share an environment (through spatial and temporal proximity), fitness effects are likely to be stronger. Consider, for the purposes of this framework, individuals that exploit the environment (scroungers, S) and those that invest in the environment or at least refrain from exploiting it (producers, P). Given a stable coupling with the environment, genes coding for benefit production will gain more benefits in future time periods than genes for exploiting the environment. Thus, the local environment changes in response to the individual’s behavior (Figure 1.1a), leading to different optimal behaviors for Ss and Ps. Producers benefit from continued coupling with the environment (positive temporal assortment) while scroungers benefit from changing spatial location (negative temporal assortment), allowing for escape from the negative temporal effects of continued coupling with a degrading environment (Figure 1.1d). The notion that exploiters are more likely to move in space than producers is
27 neither new nor controversial, but, when it is considered in the context of agents that are embedded in a spatial and temporal environment, it can lead to complex feedback loops. For instance, if individuals can process information from the environment, leaving exploited regions, spatial associations of cooperators emerge (Pepper & Smuts 1999; Pepper & Smuts 2002). Essentially, the degradation of the environment contains information about the temporal effects of coupling with the local environment, and an adaptive response to that information can be movement in space, which can generate a change in spatial assortment. Likewise, organisms can respond to spatial assortment information in a way that creates a change in temporal assortment. Passive spatial assortment (Figure 1.1b) is often the result of reproduction with limited dispersal. Individuals can respond to spatial assortment information in the environment (for example, recognizing the spatial proximity of kin) and selectively transmit benefits (or abstain from emitting costs) based on this information. When individuals change their propensity to invest in or exploit the environment, this can have long term effects on the payoffs from coupling with that environment. In other words, the processing of spatial assortment information can lead to active temporal assortment (Figure 1.1c). The potential for these complex feedback loops suggests that subtle assortment could potentially have large effects on social and spatial dynamics. Models of the evolution of cooperation reviewed for this thesis typically involve more than category of assortment (see Table 1.2), but interactions between different types of assortment are unlikely to naturally emerge unless realistic assumptions are made about the temporal and spatial behavior of agents. The primary goal of the SIMPLE (Simple agent Interaction
28 a) Passive temporal assortment
c) Active temporal assortment
b) Passive spatial assortment
d) Active spatial assortment
Figure 1.1. The diagrams above are representations of the ways in which assortment can be achieved. Blue represents productive agents, red represents scrounger agents and green represents the local physical environment. a) Passive assortment through temporal assortment can be represented as a two-way arrow between a productive (P) or energy storing (S) agent and their respective environments. b) Passive spatial assortment is shown as a one-way arrow from a P or S agent to another agent of the same type. c) Processing the signals (dashed lines) of spatial assortment leads to active temporal assortment via contingent benefit transmission. d) Processing of signals of temporal assortment leads to active spatial assortment through movement or restructuring of the physical/social environment (purple arrow).
via Movement and Production in a Local Environment) model is to realistically model spatial and temporal agent behavior with minimal assumptions, allowing for the natural
29 emergence of decision making rules that make use of information in the environment and promote assortment.
1.5. AGENT-ENVIRONMENT INTERACTION IN BEHAVIORAL ECOLOGY Movement is likely to have played an important role in the evolution of cooperation and the evolution of behavior. Not only does movement allow for individuals to pursue beneficial and even strategic behavior, it also can generate demographic effects that have widespread implications for evolutionary dynamics. Agent based modeling can be a valuable tool for addressing questions about the evolution of social behavior, providing advantages over analytical models in certain contexts. According to the gene-centered view of evolution, a gene coding for a particular behavior will be selected based on the net fitness effects on copies of the gene averaged over all contexts. However, the average fitness approach is insufficient when local interactions make the spatial and temporal location of genes relevant for determining the evolutionary dynamics (as reviewed by Wilson & Wilson 2007). When individuals interact with their physical and social environments, spatial and temporal dynamics can quickly become to complex to model analytically. If two or more of the following conditions are met, agent based simulations provide notable advantages over analytical models 1) when modeling cognition or decision making computationally/algorithmically, 2) when the heterogeneity of individual decision rules and characteristics makes it difficult to predict the aggregate effects with analytical models, 3) when events that are
30 distant in space and/or time have persistent influences 4) when spatiality and/or movement are important features of behavior and/or reproduction, and 5) when population structure and evolutionary dynamics are influenced by individual movement. First, agent based simulations are useful for modeling the individual decision making process and the computational mechanisms underlying decision making. The computational approach to the mind posits that the brain is made up of a set of information processing devices that operate on inputs from the world (Fodor 1983), and adaptationist approaches to understanding the evolution of behavior in humans also focus on the level of individual cognitive processes (Cosmides & Tooby 1992). Standard analytical models are not well-designed for representing interacting organisms that actively process information from the physical and social world. Agent based models, on the other hand, allow for the direct programming of individual-level decision rules and the exploration of the emergent effects from interactions among individuals and their environments based on those decision rules. This leads naturally to the second point, which is that agent based models allow for the modeling of heterogeneity of decision rules and characteristics. Traditional analytical models focus on the aggregate dynamics, rather than individual decisionmaking, making it difficult to model heterogeneity of population characteristics and interactions. Agent-based models, on the other hand, are well suited for modeling systems for which these features are important. When individuals are heterogeneous in their characteristics or decision rules, mathematical models of individual interactions can become intractable, making agent based simulation a valuable alternative (Hammond & Axelrod 2006; Todd et al 2005).
31 Agent based models should be preferred to analytical models the third case, where individuals have effects on one another that are often distant in space and time, as is the case in the natural world. For example, foraging animals often exploit a resource patch, affecting the availability of resources from that patch in the future. This can also be the case in social interactions where individuals use a history of interactions with a given partner or a variety of past partners to determine behavior. Hammond and Axelrod (2006) recently discussed the value of agent based simulations in these contexts, stating that formal analytical tools such as ESS techniques are insufficient when fitness “depends not just on the characteristics of the individual, but also on events distant in time and space that helped determine the individual’s current social environment.” Agent based simulations allow for the modeling of the very interactions and effects that are likely to promote cooperation, yet are analytically intractable. Spatiality and movement can be important components of individual behavior more generally; this is the fourth situation where agent based modeling can be valuable. All living organisms are embedded in a spatial environment, and they inevitably effect their local environments through the consumption of local resources and the production of beneficial or harmful byproducts. Evolution can respond these effects or select for individual abilities to respond to these effects, as is the case with the Walk Away strategy (Aktipis 2004) and foraging rules (MacArthur & Pianka 1966; Stephens & Krebs 1986). Local interactions between individuals that have spatial or network components are better suited to agent-based approaches than standard mathematical approaches because agentbased modeling environments often explicitly include space (for example, modeling individuals interacting on a lattice) and allow individuals to engage in local interactions
32 in a computationally simple manner. Further, reproduction has an inevitable spatial component in nature and agent based simulations allow this to be modeled realistically. Dispersal, migration, colonization and invasion events necessarily involve a spatial component and require individual movement in space, a point that has not received adequate attention in past models of the evolution of cooperation. These are examples of the fifth situation in which agent based models can be important: when population dynamics and subsequent evolutionary dynamics are influenced individual movement in ways that cannot be modeled analytically. Past models have neglected to consider that "the very nature of an invasion event requires active movement in space..." (Ferriere & Michod 1995). More generally, population level phenomena can be driven by individuals responding to their immediate physical and social environments, and such demographic changes can in turn influence evolutionary dynamics, including selection for cooperative behavior as well as movement. A more thorough understanding of the role of movement in the evolution of behavior could fundamentally change analytical approaches within population biology. Modeling the decision making rules used by organisms within a population could greatly increase our understanding of the interplay between individual characteristics, demographic dynamics and the evolution of behavior. Analytical models are insufficient for these purposes, and agent based models offer an effective alternative, allowing for the modeling of individual decision rules and heterogeneous populations embedded in space. Agent based models are likely to become more widespread as their value for investigating these complex questions is better appreciated. Further, agent based models are potentially invaluable tools for investigating the evolution of cooperation because
33 they combine environmental structure and individual abilities, the very factors that make analytical models intractable. The SIMPLE model described below is an example of such a model, and one that can be used to investigate phenomena that meet all five criteria: individual level computational rules, heterogeneity of characteristics, persistent spatial/temporal influences, reproduction involving spatial behavior, and the role of movement behavior in changing population structure and evolutionary dynamics. The present demonstration, however, focuses on the use of the SIMPLE model to demonstrate the value of assortment as a conceptual framework for understanding the evolution of cooperation.
1.6. A ‘SIMPLE’ MODEL OF ACTIVE ASSORTMENT Both spatial and temporal assortment rely on interaction with the physical or social environment. For the sake of tractability, analytical models typically do not embed individuals in space or in a social network. However, these are the very components that are likely to resolve the apparent paradox of the evolution of cooperation. When agents are embedded in a spatial environment, events that are distant in space and time can have important effects on the outcome for a given individual. This can be represented as a sphere of influence that expands through space as time increases, or for simplicity as a wedge in 2-dimesional space, as illustrated in Figure 1.2a. An agent based approach is particularly useful for modeling effects that individuals can have on others that are distant in time and space (Hammond & Axelrod 2006). In order to demonstrate these basic concepts and demonstrate the role of movement in promoting spatial assortment, a new model called SIMPLE (Simulation of
34 agent Interaction through Movement and Production in a Local Environment), is introduced. In this model, energy produced by agents diffuses through space (see Wakano 2007 for a similar approach). Blue agents produce energy in their local environment (producers, P) while red agent take in energy from the local environment and store it (scroungers, S), incurring a small per time period tax (see Appendix A for methodological details). Because energy diffuses through space, individuals affect their neighbors to a greater extent than those further away. In the natural world, individuals that share a local environment can have effects on each others’ fitness, such as territorial animals or plants with competing root systems. The idea that individuals affect neighbors more than those more distant in space was the motivation behind the idea of ‘ecological demes’ (also termed ‘trait groups’) (Wilson 1977). In the SIMPLE model, the positive and negative effects that neighbors can have on one another are represented as the diffusion of energy from the focal patch to neighboring patches. This energy can be conceptualized as any emitted benefit. These benefits could be physical products (such as nutrients or heat) or social products such as predator vigilance or shared information about rich foraging areas. The parameters of this model can be adjusted to model the spatial, temporal and social components of specific types of emitted benefits, but for the purpose of the present demonstration, diffusion between patches is kept constant. The environment is represented by the pink background, and this environment can be invested in (turning it lighter pink) or exploited (turning it darker pink). As time increases in the model, the one-dimensional world progresses downward and the diffusion of energy can be seen as a change in the color of neighboring patches. Agents
35 are placed on this environment and interact with the environment by consuming energy and producing energy. Red agents degrade the environment, creating an expanding wedge of dark pink/black, while blue agents increase the energy in the environment, producing a light pink/white wedge representing the increasing energy in the environment. In a simple case, red agents fully exploit their local environments and die (the end of the red lines) while blue agents survive indefinitely (Figure 1.2b). Producers benefit from long term coupling with the environment while scroungers incur costs from inhabiting a degraded environment. If the density of agents is increased (Figure 1.2c) and reproduction is enabled (Figure 1.2d), it becomes possible to see the effects of spatial assortment, with defectors in close proximity to one another dying more quickly than those in close proximity to cooperators. Other variants of this model demonstrate that movement in space can play an interesting role in the viability of cooperation. The basic results provided below are intended as demonstrations of these basic phenomena and are a starting point for more complex models. The fundamental concept shown in Figure 1.3 is that movement relative to the environment leads to a change in the coupling of individuals with their environments and therefore a change in the long-term (i.e., temporal) effects of an individual’s productive behavior. This can have three main effects on the evolution of behavior, 1) a change in the selection pressures on the productive behavior due to decoupling of the behavior with the environment, 2) a change in selection pressures
36 a)
b)
c)
d)
Figure 1.2. a) The influence of an individual can be thought of as a wedge with an increasing size as time progresses. b) A screen shot from the SIMPLE model illustrates an energy producing strategy (blue) and an energy storing strategy (red) having positive (white) and negative (black) effects, respectively, on the local environment (shades of pink) as time increases and the simulation progresses downward. Blue agents have positive effects due to temporal assortment from extended coupling with the environment, while red agents have negative effects. c) When density is higher, spatial assortment effects can be seen from the fitness
37 effects that neighbors have on one another. d) When reproduction is enabled, these effects due to spatial assortment are even more apparent.
for non-contingent movement because of the effects of decoupling, and/or 3) a change in selection pressures for responsive movement that strategically generates decoupling or coupling based on information from the environment. The first two types of effects result from selection on passive generators of environmental decoupling, and the third is an example of selection favoring active information processing and contingent responses. The three categories of effects are discussed below and examples are provided that illustrate these fundamental changes in selection pressures. The first type of effect can be understood in the context of environments that have a high degree of inherent movement of the environment (e.g., bodies of water). Consider individuals that are stationary with regard to this moving environment. With movement relative to the environment the long-term positive or negative effects of behavior can become decoupled from the local environment, because that local environment is constantly changing. A simple result of this would be a decrease in the fitness of a productive strategy that emits benefits into the local environment and an increase in the fitness of a strategy that exploits the environment (in order to collect energy to store with its soma). This is also the case with animals that collectively move in relation to their environments in a single direction, as in seasonal migration. Such a situation can be represented in the SIMPLE model, when agents move in one direction relative to the environment (Figure 1.3a). In this demonstration, relative movement can be seen to decrease the survival of productive agents (blues) increase survival of scroungers (reds) relative to a non-movement simulation.
38 The second potential effect is a change in selection pressures for movement due to the change in temporal assortment that results from decoupling. The simulation screen shot in Figure 1.3b demonstrates changes in the survival of moving vs. stationary agents. Movement relative to the environment benefits scroungers (red) while imposing costs on producers (blue). This results in greater survival of scroungers that use movement because of the positive effects of decoupling from the environment, and shorter survival of producers that use movement because producers invest in their local environments. This unsurprising result is of little value in isolation, but it demonstrates an interesting principle: selection favors movement in organisms that exist in (self and other generated) environments in which movement is advantageous, without requiring any individual level information processing abilities. A similar effect, the linkage between genes for cooperation and movement in and out of groups, has been reported in other work (Avilés et al 2004). Interestingly, a contingent movement rule to leave local environments with insufficient benefits (the Walk Away rule) creates a similar movement pattern as that which emerges from the evolution of a linkage between movement and exploitation of the environment (Figure 1.3c). This contingent rule creates adaptive behavior in both producers and scroungers, leading to the third effect, the likely selection of contingent rules that promote optimal coupling/decoupling with the environment. This demonstrates a similar principle to that shown by Pepper (2007), that the same movement rules can lead to positive assortment when used by cooperators and negative assortment when used by defectors. The Walk Away strategy is an information processing strategy; agents take in
39 information about the benefits being transmitted from the social environment and move if those benefits fall below a threshold. The Walk Away rule can formally be represented by the conditional rule below: If (E < T), Then FD1 Where E = current energy available, T = the threshold of the agent and FD1 is the movement command, ‘forward 1.’ In this decision rule, E is the information being processed from the external environment, the inequality E < T is the decision rule and FD1 is the output of the system. A benefit approach rule that allows an individual to follow a positive gradient is another simple information processing rule. Agents using this rule compare the energy in their present patch with the energy in the patch ahead, moving forward if the energy level is higher in the patch ahead. Scroungers using this rule approach producers, who might then move to ‘escape’ from the effects of neighboring scroungers and ‘find’ other producers (Figure 1.3d). The benefit approach rule can formally be represented by the conditional rule below: If (E < E1), Then FD1 Else TRN Where E = current energy available, E1 = the energy on the patch ahead, and FD1 is the movement command, ‘forward 1,’ and TRN is the movement command ‘turn around’ (allowing the organism to repeat this rule facing a different direction in the next time period). In this decision rule, E and E1 are the information being processed from the external environment, the inequality E < E1 is the decision rule and FD1 or TRN are the outputs of the system.
40 a)
b)
c)
d)
Figure 1.3. a) Directional movement in relation to the environment favors scroungers. b) Producers who move die off and those that stay survive, while scroungers who move survive and those that stay die. c) When all agents follow a Walk Away rule, producers only survive when there is a neighboring producer. d) When all agents follow a benefit approach rule, producers end up in closer spatial proximity to other producers.
41 The ability to avoid environments that impose costs or provide insufficient benefits (Walk Away) and the ability to approach benefits are nevertheless very simple rules, and rules that can provide tremendous benefits to organisms that use them. For the purposes of demonstrating fundamental concepts underlying temporal and spatial assortment, only basic results are provided as demonstrations. However, these same basic rules can be implemented in models that include reproduction, immigration, different degrees of dispersal and even environmental uncertainty. Exploratory simulations along these lines suggest that the SIMPLE model will be valuable for investigating a wide range of biological and social phenomena.
1.7. DISCUSSION It is a fact of population biology that the evolution of any trait is due to the increase in copies of the gene coding for that behavior in the population. Ceteris paribus, the effect the gene has on the fitness of ‘itself’ is no more important than the effect on copies of itself. Indeed, positive effects that accrue to a multicellular organism can be thought of more specifically as positive effects only on genes in the germ line. This kind of cooperation between cells has been considered as one of the major factors in evolutionary transitions in individuality (Maynard Smith & Szathmary 1995; Michod 1999). Despite the intuitive attractiveness of drawing categorical distinctions between ‘kinds’ of cooperation that benefit entities such as kin, group members, long-term partners or the future self, this intuitive ontology has obscured the central importance of
42 the underlying genetic reality: that genes often do not code for behaviors that increase fecundity of themselves directly, instead affecting copies of the genes (Hamilton 1964a; b; Hamilton 1975). The framework proposed here suggests that there are two main ways that genes can affect copies of themselves: through spatial and temporal proximity. Models of the evolution of cooperation can be classified according to the roles of spatial and temporal assortment, and the capacity of individuals to behave contingently results in the possibility of active assortment, where information from the environment is processed in ways that can create feedback between spatial and temporal assortment. The distinctions between passive/active assortment and spatial/temporal assortment form the basis for the present framework for classifying models of the evolution of cooperation. Many organisms are likely to have evolved the ability to contingently respond to the environment because of the benefits that can be gained from contingent behavior. Movement away from cost producing entities toward benefit producing entities, and lack of movement when in the presence of entities producing sufficient benefits, are likely to be among the first behaviors that evolution acted upon. Extremely simple organisms can use responsive movement to avoid toxins and approach energy-rich regions, suggesting that these rules can be used by organisms without a nervous system, probably through functions executed by regulator regions of the genome. Selection for simple contingent behaviors can have important effects at the population level, leading to spatial and/or temporal assortment of individuals who emit benefits, and subsequent changes in selection pressures. Contingent benefit transmission is the focal point of many proposed mechanisms promoting the evolution of cooperation including kin recognition, reciprocity and tag-based systems; however, the importance of
43 contingent movement has received comparatively less attention in the literature on the evolution of cooperation. Contingent movement plays an important role in both social and non-social aspects of behavior. Foraging adaptations are based on rules for evaluating current and future outcomes of staying in the local environment or moving to a new environment (MacArthur & Pianka 1966; Stephens & Krebs 1986). Further, migration between groups, colonization of new environments and invasion of other groups all require movement in space. The present approach suggests that the evolution of cooperation (or more specifically, benefit transmission) is likely to be dependent upon complex spatial and social dynamics such as these. The ability to process complex spatial and temporal information in ways that lead to adaptive behavior is addressed in diverse approaches to understanding learning and memory in humans and other organisms. If an organism has ability to represent and act on information about the availability of benefits that result from various rates of movement between spatial locations, this can lead to adaptive decision rules which might vary by domain (Gallistel 2000; 2002). Interestingly, the response of organisms to new information about benefits is an important component the well-known Rescorla-Wagner model of conditioning, where organisms change behavior based on the discrepancy between the rate of benefits/costs in previous time periods and the present trial (Rescorla & Wagner 1972). Research on human decision making suggests that humans can process complex information from the physical and social environment, often using heuristics that operate on regularities in this information to produce adaptive behavior with lower computational
44 requirements than alternatives (Gigerenzer 2000; Gigerenzer et al 1999). Future instantiations of the SIMPLE model could be used to investigate the evolutionary viability of simple rules in environments with different features. For example, the variability of resources in the environment or the uncertainty associated with losing investment in the local environment can be varied and the evolutionary viability of various behavioral rules that process information from the local environment can be explored with this model. Preliminary results suggest that variability of the environment can have interesting effects on the evolutionary dynamics underlying movement rules and benefit production rules. Another area that can potentially be conceptualized in the context of the present approach is that of host-parasite/pathogen interactions. Hosts are the local environments for parasites and the exploitation of that local environment influences the long-term viability of the parasite (through negative temporal effects). However, parasites and pathogens that can move to new hosts (e.g., Walk Away from exploited hosts) so they are able to escape the temporal effects of their past behavior by colonizing a new local environment. This simulation presented here can model parasite-host interactions as a special case of agent-environment interactions in which certain agents act as the local environment for other agents. There are clear parallels between the dynamics underlying parasites’ colonization of new hosts and optimal foraging models for moving between patches (Cook & Hubbard 1977). These combined features make the SIMPLE model potentially suitable for modeling such interactions. Because the spatial proximity of both hosts and parasites can influence the costs and benefits to each actor as well as the likelihood of transmission, spatial models with contingent agent behavior might provide
45 an advantage over analytical models for understanding some host-parasite dynamics that emerge from subtle spatial effects. Other topics may be understood in the context of the present framework. For example, the notion of epigenetic effects (where a gene coding for a beneficial trait at one point in time have negative effects at another time) can be conceptualized in terms of temporal assortment. The phenotypic ‘vehicle’ (Dawkins 1976/1989) created by a set of genes can be thought of a part of the local environment with which the genes have an obligatory long-term association (until reproduction and/or death). This long-term coupling of the genes with the vehicle results in temporal assortment and the potential for tradeoffs between different time periods. Along similar lines, it might be the case that life history and reproductive mechanisms are designed to enable contingent movement of genes into the optimal vehicles (in terms of size, number and other characteristics) based on information from the environment about resource availability (temporal assortment) and the effects from proximity of kin (spatial assortment). The suggests interesting connections to parental investment theory (Trivers 1972) and the ideas of r vs. K selection (MacAurthur & Wilson 1967).
1.8. CONCLUSION Analytical models are limited in their abilities to model fitness effects due to subtle temporal and spatial effects (Hammond & Axelrod 2006), and the modeling of feedback between temporal and spatial assortment adds an additional layer of complexity. An agent based approach, as exemplified by the SIMPLE model, provides a
46 demonstration of a number of phenomena that emerge from active information processing on the part of agents, including changes in spatial assortment. The ultimate value of the framework for the evolution of cooperation presented here is that it provides a concise set of principles that distinguishes among evolutionary effects, individual benefits and contingent behavior (in contrast to Bergmuller et al 2007a; Lehmann & Keller 2006; Nowak 2006; Sachs et al 2004) The present system classifies a large number of models of the evolution of cooperation as manifestations of a few general concepts: assortment of genes in space or time and through passive or active processes (Tables 1.1 and 1.2). The essence of active assortment is the ability to move or change benefit transmission based on information from the environment. Complex feedback loops can emerge from active assortment because agents can act on environmental information (such as the availability of benefits or likely proximity of kin) in ways that change the information in the environment. Individuals can respond to insufficient benefits (temporal assortment information) by moving in space (changing spatial assortment information) or respond to the proximity of likely kin (spatial assortment information) by emitting benefits (changing temporal assortment information). The possibility of these feedback loops suggests that the present framework and the SIMPLE model might provide unique insights and traction for investigating interrelations among behaviors involving movement and/or benefit transmission. This framework is not only valuable for categorizing the mechanisms that underlie the evolution of cooperation, it also suggests connections among diverse biological and social phenomena that make use of the temporal and spatial association of genes (as well as information about these
47 associations), including, but not limited to, local reproduction, offspring dispersal, colonization of new areas, foraging adaptations, the contingent transmission of benefits, and changes in social organization through responsive movement.
1.9. OVERVIEW OF REMAINING CHAPTERS The simulation results presented in this dissertation indicate that responsive movement allows cooperation to be a viable strategy. This includes results from dyadic interactions (Chapter 2), where the cooperative “Walk Away” strategy outperformed Titfor-Tat under a variety of parameter values, and results from group interactions (Chapter 3) where cooperators with sufficiently high thresholds (‘expectations’ for the level of cooperation within a group) outperformed defectors. The importance of responsive movement for the evolution of cooperation is discussed (Chapter 4), focusing on the ways in which responsive movement can affect population dynamics, potentially increasing the effectiveness of multilevel selection. The basic goal of this research program is to build sociality from the ground up, starting with the most basic components of behavior and adding additional components after the aggregate behavior of simple strategies (such as “Walk Away”) are well understood. The last chapter (5) outlines the goals of this program and describes a number of additions and modifications to the “Walk Away” project that will serve to carry forward this research program and apply the theoretical and agent-based tools to issues in theoretical biology, psychology, sociology and public policy.
48
CHAPTER 2 KNOW WHEN TO WALK AWAY: CONTINGENT MOVEMENT AND THE EVOLUTION OF COOPERATION
2.0. ABSTRACT Models of the evolution of cooperation suggest that an important characteristic of successful strategies is the ability to respond contingently to the social environment. A number of mechanisms by which this can be accomplished have been suggested, some of which require relatively complex information processing systems. This research explores relaxing the requirements on information processing while preserving the evolvability of a cooperative strategy. The agent-based computer simulations reported here show that ‘Walk Away,’ a behavioral rule of extremely limited complexity (move after partner defects), can outperform more complex strategies under a number of conditions. Previous simulations of exit strategies have not examined the effect of implicit and explicit movement costs, different error rates, or the simultaneous presence of TFT and PAVLOV. The simulations reported here establish that the Walk Away strategy resists invasion and can invade a population of defectors at a lower initial frequency than any other strategy. The Walk Away strategy was successful, despite its simplicity, because it exploited aspects of the physical and social environment.
49
2.1. INTRODUCTION For decades, evolutionary biologists and game theorists have pondered the origins of cooperation, aided by the powerful tools afforded by the game theoretical approach (Axelrod 1984; 1997; Trivers 1971). Such work has illustrated that simple strategies such as Tit-for-Tat (TFT) can be successful. Of course, the success of a given strategy depends on the frequency of other strategies in the social environment, an idea that is borne out in computer simulations of the evolution of cooperation (Axelrod 1984; 1997; Trivers 1971). In most past work, strategies interact with one another according to a stochastic matching process, simulating random encounters with others in the physical world. However, for many species, encounters with others will not only be non-random, but depend systematically on the way that organisms move in space. Because assortment can promote the evolution of cooperation (Wilson & Dugatkin 1997), spatial models have much to offer. Recently there has been increased interest in spatial models among a number of researchers (Brauchli et al 1999; Ferriere & Michod 1995; 1996; Killingback & Doebeli 1996; Nowak & May 1992) including evolutionary psychologists (Kenrick et al 2003; Kenrick et al 2002). In the current model, space is simulated as a lattice of occupiable patches, populated with multiple mobile agents. The simulations reported here describe the performance of a cooperative strategy that uses contingent movement to avoid repeated interaction with defectors. The success of this strategy is explored under a variety of parameter values and with a number of competing strategies such as TFT and PAVLOV.
50 2.1.1. Simple strategies Despite the existence of more complex processes in humans (such as memory for repeated interactions, punishment of defectors and reputation), cooperation might have originally evolved through much simpler mechanisms. Similarly, cooperation in other animals might be realized through relatively simple decision rules. As Anatol Rappoport’s TFT strategy (Axelrod 1984) and Nowak and Sigmund’s PAVLOV strategy (Nowak & Sigmund 1993) illustrate, a simple strategy can outperform more complex strategies. TFT’s simplicity lies in the fact that it simply copies the last behavior of its partner. PAVLOV uses a ‘win stay, lose shift’ strategy, meaning that it switches its behavior (from C to D or from D to C) whenever its partner defects. In the simulations reported here, the viability of another simple strategy, ‘Walk Away,’ is examined. This strategy could be described as ‘win stay, lose move,’ and it is different from TFT and PAVLOV in that it always cooperates and it moves away from a defecting partner instead of changing its behavior.
Cooperate
Defect
Cooperate
3,3
-1,5
Defect
5,-1
0,0
Figure 2.1. Prisoner’s dilemma payoff matrix (row player is in bold).
51 The simulations reported here and elsewhere use the prisoner’s dilemma paradigm (Fig. 2.1). Both individuals are better off if they both cooperate than if they both defect, but each individual would be better off defecting (because defection has a higher payoff no matter what the other player does). In Axelrod’s (1984) round-robin simulation he found that the cooperative strategy submitted by Anatol Rappoport, TFT, outperformed all other strategies. TFT was well suited for the environment in which it competed, but the environment of Axelrod’s simulations abstracted from the real world in several important regards. Most relevant to the work described here, the environment in these simulations was not spatial (as the real biological world is) and the interactions took place in a somewhat unrealistic round-robin fashion (which is difficult to interpret spatially). The work reported here aims to create a more realistic environment in which agents can interact.
2.1.2. Game-theoretic models Axelrod’s round-robin tournament is only one of several classes of simulations. Some simulations do not represent space at all (Axelrod 1984; 1997; Nowak & Sigmund 1993), and others represent space implicitly by including search cost (Dugatkin 1992; Dugatkin & Wilson 1991; Enquist & Leimar 1993; Eshel & Cavalli-Sforza 1982; Peck & Feldman 1986). Spatial game theoretic models represent space explicitly, but do not necessarily involve movement in space (Brauchli et al 1999; Killingback & Doebeli 1996; Nowak & May 1992). For these models, spatiality is only relevant in that successful
52 strategies can expand into areas previously occupied by other strategies; the individuals themselves never move. Another class of simulations, agent-based simulations, are explicitly spatial and can involve movement in space (Ferriere & Michod 1995; 1996; Pepper & Smuts 1999). One of the advantages of using spatial agent-based models is that they can approximate many of the features of the biological and social world in which we (and other animals) evolved, while providing the opportunity to carefully control and monitor the changes in the system over time. In most models, whether they are spatial or not, all individuals have the same number of total interactions. In the present model, this is not the case; agents move in the spatial world until they encounter another agent. After interacting once, they either continue moving or stay on the patch. If both individuals stay, they will continue to interact. If one or both individuals leave, no further interaction takes place between those two individuals.
2.1.3. Spatiality and movement Previous research has shown that spatiality can favor cooperation because it generates assortment, resulting in the “conspiracy of cooperators” (Brauchli et al 1999; Ferriere & Michod 1995; 1996; Killingback & Doebeli 1996; Nowak & May 1992), although this depends on the payoff structure (Hauert & Doebeli 2004), the initial parameters and the level of stochasticity (Hauert 2002). In that work, unlike the present research, individuals did not move in space. Instead, all patches were occupied by individuals who interacted only with neighbors. Changes in the population structure are
53 determined by replacing each individual with the most successful strategy among the nearby cells (Nowak & May 1992), or by similar rules that involve imitating neighbors in proportion to their relative success (Hauert 2002). These models suggest that spatiality favors the evolution of cooperation, or at least limits the ability of defectors to take over a population. In contrast, for models with mobile individuals, the conclusion has generally been that mobility constrains the evolution of cooperation because defectors can find cooperators to exploit (Dugatkin 1992; Dugatkin & Wilson 1991; Enquist & Leimar 1993). These models focused on how defectors can use movement for their own gain, but not how cooperators might use movement for their own gain. In the present study, cooperative agents used a contingent movement strategy to avoid repeated interactions with defectors.
2.1.4. Exit and contingent cooperation Exit strategies, such as ‘out-for tat’ (Yamagishi et al 1994) are similar to the Walk Away strategy in that they use information about a partner’s previous behavior to determine whether or not to interact with that partner. However, the exit strategies explored in other simulations often assumed a fairly complex agent (Vanberg & Congleton 1992; Yamagishi et al 1994), or imposed an unrealistic scheme of reassigning agents to another partner immediately after exiting an interaction (Schuessler 1989). In the present study, it is shown that physical movement away from defectors can substitute for behavioral complexity.
54
2.2. STRATEGY DESCRIPTION By exploiting space, the Walk Away strategy obviates memory and recognition. The Walk Away cooperator moves through space and interacts when it encounters another individual. If its partner cooperates, Walk Away stays on that same patch, but if its partner defects, Walk Away moves. This enables Walk Away to engage in repeated interactions with cooperators and avoid repeated interactions with defectors without using memory. This strategy requires only two states and four transition rules (Fig. 2.2).
Figure 2.2. State transition figures show the simplicity of the Walk Away strategy. Boxes indicate the possible states the agent can occupy and the arrows show the possible transitions between states. Words next to arrows indicate the partner behavior associated with each transition. Walk Away and Naïve are pure strategies with all-C and all-D versions. When partner behavior on arrow is ‘nothing,’ this indicates the state change rule for when there is no partner.
In the first simulation, this strategy is pitted against three other simple strategies of equivalent complexity. One is a defecting Walk Away strategy, which employs the same rules but always defects instead of always cooperating (essentially preying on cooperators). Also included were naïve cooperators and defectors, which move when
55 they are without a partner and stay when they find a partner regardless of that partner’s behavior (Fig. 2.2). After comparing the performance of Walk Away to other moving strategies of minimal complexity, it is then compared to the well-known Tit-for-Tat (TFT) strategy (Axelrod 1984). TFT follows the simple rule of copying the last behavior of its partner. Two versions of TFT were implemented in these simulations, one was a mobile TFT and the other was a stationary TFT. Mobile TFT moves in the environment when it is without a partner and stays when it finds a partner, employing the ‘tit-for-tat’ rule. This strategy is more computationally complex than ‘Walk Away,’ requiring three states and nine transition rules. Stationary TFT was included largely because is equivalent to Walk Away in its computational complexity (Fig. 2.2). Walk Away is also compared to the PAVLOV strategy, another simple strategy that is successful in certain environments (Nowak & Sigmund 1993). PAVLOV uses a ‘win-stay, lose-shift’ strategy, which means it switches its strategy whenever its partner defects. This enables it to exploit cooperators but leaves it fairly vulnerable to defectors (because it continually switches back and forth from D to C when it is with a defector). PAVLOV also has the capacity to ‘correct’ mistakes, avoiding cycles of retaliation that occur with stochastic TFT players, making it more resilient to noise than strategies like TFT (Nowak & Sigmund 1993). PAVLOV was also implemented as both a mobile strategy (requiring three states and nine transition rules) and stationary strategy (requiring two states and four transition rules; Fig. 2.3).
56
Figure 2.3. State transition figures show both versions of TFT and PAVLOV used in these simulations. The boxes indicate the states the agent can occupy and arrows show the possible transitions among states. The ‘testing’ behavior of PAVLOV is not embodied in this figure (see sections 2.2 and 2.4.2). When partner behavior on arrow is ‘nothing,’ this indicates the state change rule for when there is no partner.
2.3. METHODS (SIMULATION DESCRIPTION) In this simulation, energy was the currency that determined reproduction and death. There was no other source of ‘energy’ besides the interactions with other agents and the initial energy of each agent. At the beginning of the simulation, agents started with an energy level chosen from a uniform distribution between 0 and 49. When the
57 energy of an agent reached 0, the agent died. If the energy of the agent reached 100, that agent reproduced without mutation, creating a copy of itself and splitting its energy with its offspring. Offspring were placed on a random patch. At the beginning of each simulation, 25 agents of each type were introduced (unless otherwise specified). A carrying capacity equivalent to the starting number of agents for each simulation was implemented to ensure that the simulations would run in a reasonable amount of time. In order to keep the number of initial agents of each type the same across simulations, the initial number of total agents (and therefore the carrying capacity) differed across some of the simulations. When the carrying capacity of the environment was reached, a random agent’s energy was decreased by 10. These random energy decrements continued until enough agents died and the population was again within the carrying capacity. Occasionally, the total number of agents was lower than the carrying capacity (as can be seen in Figures 2.4 and 2.5) because agents died both when their energy was decremented by the carrying capacity subroutine and when their energy reached 0 due to their interactions with partners. The number of agents of each strategy usually achieved stability within 500 time steps, and the length of all runs was 1000 time steps. Agents inhabited a spatial world with 25 x 25 unique patches that could be inhabited by one or more agents. The patches wrapped from bottom to top and left to right, forming a torroidal lattice. All simulations were implemented in Starlogo 1.2.2 (MIT Media Laboratory 2001), an agent-based programming environment. In each simulation, agents moved randomly, moving forward and changing their heading randomly to the left or right during each time step. When two agents encountered
58 each other, an interaction took place and payoffs were assigned according to the prisoner’s dilemma payoff matrix (Fig. 2.1). Agents were updated in virtual parallel; during each time step each agent had one interaction with another agent. After this interaction, each agent could choose to stay on the current patch or move to a neighboring patch. Agents either stayed in the same patch unconditionally, or stayed only if their partner cooperated. When both partners stayed on the patch, they interacted again in the next round. If another agent landed on a patch with two interacting agents already on it, two of the three agents on that patch would be randomly paired. This means that an agent could potentially split up an interacting dyad by interacting with one of the agents in the existing pair. In order to examine how robust a strategy is to ‘mistakes,’ there was a .001 probability that an agent would make a ‘mistake’ by cooperating instead of defecting, or vice versa. This error could potentially break up a cooperative pair if the partner was a contingently moving cooperator. In Simulation 7, several different error rates were studied in order to investigate the viability of Walk Away under greater noise.
2.4. RESULTS 2.4.1. Basic model (Simulation 1) The first simulation used 4 types of agents: ‘naïve’ cooperators that cooperate and stay even if their partner just defected, Walk Away cooperators that cooperate and then stay only if their partner just cooperated, ‘naïve’ defectors that defect and stay regardless
59 of their partner’s behavior, and Walk Away defectors that defect and only stay with a cooperator. The simulation began with 100 agents, 25 of each type. During a typical run of this simulation, the proportion of defecting agents (Walk Away defectors and ‘naïve’ defectors) initially increased as they exploited ‘naïve’ cooperators. This caused ‘naïve’ cooperators to decrease in number and eventually die out. With no ‘naïve’ cooperators left to exploit (and an inability to exploit Walk Away cooperators for more than one time period), defecting agents decreased in frequency. Walk Away cooperators were then left as the only agents capable of maintaining mutually beneficial interactions and they completely overtook the defecting strategies in each of the 10 runs (see Table 2.1). Figure 2.4 shows the change in number of agents of each strategy over time in a typical run.
Figure 2.4. Area plot of the number of agents of each strategy over time in a typical run of Simulation 1.
60 2.4.2. Comparative performance of TFT and PAVLOV (Simulations 2-4) The second, third and fourth simulations compared the performance of the four strategies in the basic model with two Tit-for-Tat strategies and two PAVLOV strategies. All features of these simulations were identical to Simulation 1 except for the strategies included and the total number of agents; the initial number of agents and the carrying capacity were increased to 150 (Simulations 2 and 3) or 200 (Simulation 4). Although changing the total number of agents changes the density which has an effect on the outcomes of the 8-strategy model, the density increases in these simulations are within a range that has little qualitative effect on the general outcomes (see section 2.4.6)2. 2.4.2.1. Simulation 2: Tit-for-Tat The second simulation included two types of agents that used the Tit-for-Tat strategy: one that moved when it was without a partner and one that never moved (see Figure 2.3). At the end of the simulations, most of agents were Walk Away cooperators (M=58.7%, SD=17.1%). However, the mobile TFT strategy always maintained some percentage of the population (M=24.4%, SD=10.6%). ‘Naïve’ cooperators also achieved limited success (M=14.4%, SD=14%) (See Table 2.1.). 2.4.2.2 Simulation 3: PAVLOV The third simulation was identical to the first simulation, except that it included two PAVLOV strategies. On occasion, PAVLOV attempts defection, which allows it to exploit certain cooperators. In this simulation, the likelihood of PAVLOV ‘testing’ its
2
In truth, it is not possible to fully determine this based on the simulations run in section 2.4.6. In order to do this, simulations with varying density levels would have to be conducted with the basic model as well as the models that include TFT or PAVLOV.
61 partner was .01, which replaced the lower error rate of .001 used by the other strategies. As with TFT, there were two PAVLOV strategies, one that moved unless it had a partner, and another that never moved (see Figure 2.3). Again, the Walk Away cooperator strategy attained the highest frequency in each run (M=92.4%, SD=11.7%) (see Table 1). Mobile PAVLOV also maintained a positive frequency in some runs (M=6.0 %, SD=10.6%).
Number Agents
Number of Agents of Each Type 200 180 160 140 120 100 80 60 40 20 0
Walk Away Cooperator Naive Cooperator Walk Away Defector Naive Defector Mobile TFT Stationary TFT Mobile PAVLOV Stationary PAVLOV 1
1000 Time
Figure 2.5. Area plot of the number of agents of each strategy over time in a typical run of Simulation 4.
2.4.2.3. Simulation 4: Tit-for-Tat and PAVLOV Simulation 4 included all four strategies in Simulation 1, as well as the two Titfor-tat strategies from Simulation 2 and the two PAVLOV strategies from Simulation 3, for a total of eight strategies and a carrying capacity of 200 agents. At the end of these runs, Walk Away cooperators were the most successful (M= 78.1%, SD=13.7%), with mobile TFT second (M=10.9%, SD=10.0%) and mobile PAVLOV next (M=6.8%,
62 SD=4.1%). ‘Naïve’ cooperators also had limited success (M=3.6%, SD=5.4%). The results of Simulation 4 are summarized in Table 2.1, and Figure 2.5 shows the change in frequency of each strategy over 1000 time periods in a typical run.
Number Agents, t=1000
Comparison of Invadability 100
+
80
TFT PAV
60
Walk Away
40 20 0 0
20
40 60 Number Agents, t=0
80
100
Figure 2.6. Minimum initial frequency necessary for agents to invade a population of defectors. Initial number of agents of a particular type (out of 100 total) is plotted on the x-axis and the number of agents after 1000 time periods is plotted on y-axis. Ten runs were performed for each initial frequency, but simulations were not run for initial number of Walk Away over 16 or TFT agents over 48 because it was clear that the strategies successfully invaded at these frequencies.
2.4.3. Invasion (Simulations 5-6) Simulations 5 and 6 examine Walk Away’s resistance to invasion by other strategies and the comparative ability of Walk Away to invade a population of defectors. 2.4.3.1. Simulation 5: Resistance to invasion
63 Resistance to invasion is a necessary property of an Evolutionarily Stable Strategy (ESS) (Axelrod 1984). Although the term ESS, as originally formulated, does not strictly apply to this model because this model is made up of a finite number of agents (Nowak et al 2004), the resistance of Walk Away to invasion by other strategies is nonetheless investigated here. This simulation included all eight strategies used in Simulation 4 and was identical in all respects except that every 100 time periods, five agents of each strategy were introduced (with starting energy of 50). This provided an opportunity for strategies that had died out to invade the population. However, this does not constitute a comprehensive test of invadability; such a test would need to include separate simulations to examine whether any strategies can invade Walk Away without any other strategies present. TABLE 2.1 Percentage of Agents of each Type in Simulations 1-5 [M % (SD in %)] 1
2
3
4
5
Naïve Coop
0.0 (0.0)
14.4 (14)
0.0 (0.0)
3.6 (5.4)
4.7 (3.9)
Walk Away Coop
100 (0.0)
58.7 (17.1)
92.4 (11.7)
78.1 (13.7)
76.9 (9.3)
Naïve Defect
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.1 (0.2)
Walk Away Defect
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
Mobile TFT
N/A
24.4 (10.6)
N/A
10.9 (10.0)
10.7 (8.0)
Stationary TFT
N/A
1.2 (1.5)
N/A
0.5 (0.7)
1.8 (1.3)
Mobile PAVLOV
N/A
N/A
6.0 (10.6)
6.8 (4.1)
5.2 (3.2)
Stationary PAVLOV
N/A
N/A
7.3 (0.2)
0.2 (0.5)
0.7 (0.8)
Table 2.1. Number of agents of each strategy after 1000 time periods for Simulations 1-5. Results are averaged from 10 runs. Simulations 1-4 started with 25 agents of each included type and changes in the population occurred only through reproduction and death. In Simulation 5, five agents of each type were introduced into the population every 100 time periods.
64 Results of simulation 5 are very similar to those of simulation 4. Walk Away cooperators were the clear winner (M=76.9%, SD=9.3%), with Mobile TFT next (M=10.7%, SD=8.0%) and ‘naïve’ cooperators (M=4.7%, SD=3.9%) and Mobile PAVLOV (M=5.2%, SD=3.2%) also attaining a limited frequency in the population (see Table 2.1). 2.4.3.2. Simulation 6: Invasion of cooperation Previous work has shown that TFT can invade a population of defectors if there are a sufficiently high number of TFT pairings (Ferriere & Michod 1996). The simulations in this section compare the abilities of TFT, PAVLOV and Walk Away to invade a population of defectors3. The population in all of these simulations was made up 100 agents, including some initial number of one of the following strategies: Walk Away Cooperate, Mobile TFT, or Mobile PAVLOV (indicated on the x-axis in Fig. 2.6). The remaining agents consisted of half ‘naïve’ Defectors and half Walk Away Defectors. Ten runs were carried out for each even number of initial frequencies over a range of initial frequencies (i.e., 2, 4, 6… initial Walk Away, TFT or PAVLOV agents). As can be seen from Figure 2.6, Walk Away cooperators invaded a population of defecting agents at a much lower initial frequency than either TFT or PAVLOV. Walk Away invaded the population of defectors at initial frequencies as low as 2 (out of 100), and, at initial frequencies of 8 or more, Walk Away invaded the population in all runs. In contrast, the lowest frequency at which TFT invaded the population was 18, and only at initial 3
The ability of Walk Away to invade populations of cooperative strategies (TFT and PAVLOV) was not examined because Walk Away is functionally identical to these other strategies in the absence of defectors (in that they will always cooperate (except for PAVLOV’s occasional ‘testing’). This means that there would be no directional selection in such a population of mixed cooperative strategies.
65 frequencies of 40 or more did TFT agents always invade the population of defectors. PAVLOV performed worse than both Walk Away and TFT; the lowest frequency at which PAVLOV invaded was 86, and even when the initial number of PAVLOV agents was 98, they did not always invade. TABLE 2.2 Percentage of Agents at Various Error Rates [M % (SD in %)] .001* Naïve Coop Walk Away Coop
4
2.2 (2.9)
.002
.005
.01
.02
.1
2.1 (3.5)
1.5 (2.5)
3.0 (5.0)
0.7 (1.7)
0.0 (0.0)
62.9 (20.3)
39.4 (24.8)
20.7 (33.6)
0.0 (0.0)
79.2 (10.6) 75.2 (15.7)
Naïve Defect
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
1.1 (3.5)
93.9 (10.9)
Walk Away Defect
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
Mobile TFT
13.6 (9.6)
16.9 (15.0)
26.2 (21.1)
44.5 (27.3)
68.1 (35.0)
5.6 (10.5)
Stationary TFT
0.4 (.9)
0.7 (1.1)
0.9 (1.2)
1.0 (1.9)
2.3 (3.4)
0.2 (0.7)
Mobile PAVLOV
4.6 (4.7)
5.1 (6.0)
8.5 (7.5)
11.8 (9.8)
7.0 (7.1)
0.3 (0.8)
Stationary PAVLOV
0.1 (0.2)
0.0 (.1)
0.1 (.3)
0.4 (0.9)
0.2 (0.6)
0.0 (0.0)
Table 2.2. The number of agents (out of 200 total) of each type left after 1000 time periods. The most successful strategy at each error rate is in bold.
2.4.4. Error (Simulation 7) Recall that there was a .001 probability that agents would make a ‘mistake’ defecting instead of cooperating, or vice versa. Investigations of the role of error indicated that the Walk Away strategy is sensitive to error rates, doing extremely well at low levels of error and doing poorly at high error rates. When the error rate is increased, 4
All other simulations reported in this article use this error rate.
66 mobile TFT agents are most successful over an interval of error rates. At yet higher error rates, the ‘naïve’ defector strategy is most successful. Table 2.2 summarizes these results.
2.4.5. Movement Cost (Simulation 8) In the real world, there is a cost associated with moving. The viability of Walk Away with various movement costs was examined here. Movement cost was subtracted from the energy of the agent every time that agent moved from one patch to another. Walk Away was successful at only at relatively low levels of movement cost (less than 3). However, it is important to take into account the ratio of movement cost to the payoffs associated with partner interactions. The highest payoff possible from an interaction is 5 (when an agent defects with a cooperator) and the mutually cooperative payoff is only 3. It appears that the Walk Away strategy continued to be successful in the face of movement costs (at this particular inverse density level of 3.13), as long as the movement costs were less than the benefit from one round of cooperation. When movement cost was between 3 and 9, Mobile TFT was the most successful strategy. Presumably, Mobile TFT outperforms Walk Away at intermediate levels of movement cost because it stays with a partner after a defecting ‘mistake,’ while Walk Away moves after defection even if that defection is the result of error. This means that Walk Away will incur movement costs much more often than Mobile TFT.
67 When the movement cost was between 9 and 45 (movement cost of greater than 45 were not explored in these simulations), Stationary TFT and Stationary PAVLOV dominated the population in approximately equal numbers (although mobile strategies continued to make up approximately 25% of the population). Because the stationary strategies did not move at all, their success in the face of high movement costs is not surprising. However, the continued existence of mobile strategies is more surprising and the reasons for it more complex. Because agents are placed on random patches when they ‘hatch,’ stationary strategies only get partners if other agents find them. Even when the cost of moving is very high, it can still be worth moving if the likelihood of obtaining a partner as a stationary individual is sufficiently low. This results in frequency dependent selection on mobile strategies which accounts for their continued existence in the face of high movement costs.
Number Agents
Changing Density 200 180 160 140 120 100 80 60 40 20 0 0.85
Walk Away Coop Naïve Coop Walk Away Defect Naïve Defect Mobile TFT Stationary TFT Mobile PAVLOV
1.81
3.13 5.45 10.13 21.13 41.41 Patches per agent
Stationary PAVLOV Figure 2.7. Number of agents of each type at various densities. The number of patches per agent
(inverse of density) increases approximately logarithmically on the x-axis.
68 2.4.6. Density (Simulation 9) Several different densities were investigated in order to determine what density conditions cause Walk Away to lose its advantage. Ten simulations were run for 1000 time steps at several density levels. Number of patches per agent (the inverse of density) was varied, but because of software constraints and the need to keep the grid square, these values are not whole numbers. In the simulations reported in earlier sections of this chapter, the number of patches per agent ranged from 6.25 (simulations with 100 agents) to 3.13 (simulations with 200 agents). However, in the present section, a much larger range of inverse density values was applied (.85 - 41.41). At very high densities (when there was less than one patch per agent), Walk Away did extremely poorly and the defecting types were most successful. Mobile TFT also did relatively well. Although Walk Away does poorly at extremely high densities (i.e., low inverse densities) because dyads are continuously broken up by less cooperative agents, Walk Away does very well over a fairly large range of densities (see Figure 2.7). Walk Away cooperators achieved an average 88.6% of agents (SD=10.9%) when the number of patches per agent was as low as 1.81. Walk Away cooperators also did well when the number of patches per agent was much higher; at 5.45 patches per agent, Walk Away cooperators still achieved an average of 61.5% (SD=15.1%). Only when the number of patches per agent was increased to 10.13, did mobile TFT (M=49.1%, SD=15.3%) agents achieve a greater frequency than Walk Away cooperators (M=29.1%, SD=8.9%). ‘Naïve’ cooperators also faired better when the number of patches per agent was high, although this was not apparent until the number of
69 patches per agent increased to 21.1. Walk Away’s strategy involved leaving a partner after one defection, even if that defection was the result of a mistake (which occurred with probability .001). The most likely reason that the strategy did not do as well at extremely low densities is that it left these potentially cooperative and very hard to find partnerships after just one defection. In these circumstances TFT had an advantage, since it did not leave after its partner defected.
2.4.7. Increased Mutual Defection Payoff (Simulation 10) In simulation 10, the payoff for mutual defection was increased from 0 to 1. Relative success of each strategy in both the 4-strategy model and the more complex 8strategy model were investigated. Results are reported for 100 runs with a length of 1000 time steps each. In the basic 4-strategy model with Walk Away cooperators, naïve cooperators, Walk Away defectors and naïve defectors, the results over 100 runs were slightly different. In 91 runs, only Walk Away cooperators were left after 1000 time periods. Of the remaining nine runs, six ended with only naïve defectors, two ended with mostly Walk Away cooperators and a small number of naïve cooperators, and one ended with equal numbers of Walk Away cooperators and naïve defectors. In six of the runs, naïve defectors were able to interact with each other and out-compete Walk Away cooperators because of the small but positive mutual defection payoff. In the runs that ended with both Walk Away cooperators and naïve defectors, all dyads (lasting longer than one time period) were made up of homogenous individuals (i.e., either both Walk Away cooperators or both naïve defectors).
70 In contrast, results of the 8-strategy model (with TFT and PAVLOV) showed that Walk Away cooperators dominated the population in every run, averaging 75.6% (SD=17.6%) at the end of each run. Mobile TFT agents were present in the population at the end of most runs (M=19.6%, SD=16.2%), whereas defecting agents were absent at the end of every run.
2.5. DISCUSSION Walk Away agents had no memory of the action of other agents; they only responded to their partner’s most recent behavior. Despite these limited capabilities, Walk Away was successful because agents employing this strategy were able to achieve assortative interactions. Walk Away was able to reap the benefits of repeated interactions with cooperators because other cooperators stayed on the same patch and this strategy was able to avoid continued interactions with defectors by leaving the patch.
2.5.1. Contingent Movement and Assortment The movement rules employed by Walk Away resulted in behavioral assortment. When Walk Away interacted with a defector, it moved, but when it interacted with another cooperator, it stayed. The cooperative dyads that resulted from this assortment were far more stable and fecund than dyads made up of agents that did not cooperate in every round. In fact, when agents use movement rules to avoid defectors, dyads of defectors or mixed dyads will be much less stable (often existing for only one round), than groups of cooperators. In essence, the Walk Away cooperators outperformed other
71 strategies due to the effects of between-group selection. As Wilson (Wilson 1983; Wilson & Dugatkin 1997) notes, selection can act at the group level when there is behavioral assortment because this the leads to greater variation between groups. Because of the movement rule employed by Walk Away, behavioral assortment led to greater fitness of the cooperative dyads (compared to non-cooperative ones). This led to greater evolutionary success for agents employing the Walk Away strategy5. Like TFT, strategies based on altruistic punishment (Fehr & Gachter 2002; Price et al 2002), indirect reciprocity (Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003), and other types of assortment (Eshel & Cavalli-Sforza 1982; Wilson & Dugatkin 1997 ) enable the formation of a “conspiracy of cooperators,” limiting potential benefits to defectors. The PAVLOV strategy accomplishes this to a lesser extent, as defectors can exploit PAVLOV’s switching behavior.
2.5.2. Comparing Walk Away to TFT and PAVLOV In several ways, the Walk Away strategy is similar to TFT. After interacting with an agent who defects, both Walk Away and TFT stop cooperating with that individual: TFT punishes the defector by defecting in turn; Walk Away ‘punishes’ the defector by not interacting in the next time period. Because certain conceptions of the TFT strategy equate defection and non-action, often implicitly through having mutual defection payoffs of 0, 0 (e.g., Nowak & May 1992), TFT and Walk Away can be construed as fundamentally the same strategy. However, the difference between Walk Away and TFT 5
It is important to note that Walk Away does not need to form clusters of dyads in order to succeed. Dyads of Walk Away agents were instead relatively evenly distributed over the spatial grid. The only assortment taking place in this simulation was within dyads (i.e., Walk Away cooperators formed dyads with each other).
72 becomes clear by virtue of their unique instantiations in a spatial, multi-agent world. While TFT stays with a defecting partner, Walk Away leaves a partner who defects (and seeks out a new partner), no matter how many times a partner has cooperated before. After an interaction with a defector, Walk Away pursues a strategy of contingent movement (switching from stay to move) while TFT pursues a strategy of contingent behavior (switching from cooperate to defect). While Walk Away’s strategy enables it to find a new partner, TFT is essentially stuck in a partnership with whichever agent happens to be on the same patch. An additional difference between TFT and Walk Away is that Walk Away needs no memory, while TFT needs memory for at least one round of play. When Walk Away interacts with a defector, it simply responds to that defection by moving. TFT, on the other hand, must remember that interaction so it can respond appropriately in the next round (unless TFT is somehow responding to the behavior of its partner without actually remembering it). Although this point might seem trivial, the alternative conception of contingent cooperation provided by the Walk Away strategy might provide deeper insight into the evolution of cooperation in non-human animals (see section 2.5.4). Like PAVLOV, the Walk Away cooperator and Walk Away defector strategies use contingent cooperation, but instead of employing a ‘win-stay, lose-shift,’ strategy, Walk Away uses a ‘win-stay, lose-move’ strategy, never switching to defection. PAVLOV responds to the behavior of its partner by continuing to do the same thing (either cooperating or defecting) if its partner cooperates and changing state (to cooperation or defection) if its partner defects (Axelrod 1997; Nowak & Sigmund 1993).
73 Both PAVLOV and TFT differ from Walk Away in that they defect under certain conditions, while Walk Away is a purely cooperative strategy.
2.5.3. Similar Models of the Evolution of Cooperation Interestingly, the results of the current simulations largely contradict the conclusions of a similar study in which Enquist and Leimar (1993) investigated the evolution of cooperation in mobile organisms using a mathematical model. They concluded that mobility constrained the evolution of cooperation (in their simulation the cooperative strategy played TFT). However, their model did suggest that long search times (for partners) and long coalition times (i.e., the time during which a particular dyad stays together) would favor the evolution of cooperation. This suggests that the different conclusions of the simulations presented in this chapter and Enquist and Leimar’s model turn on differences in search times and the length of coalitions, both being longer in the simulation reported here. Indeed, the results reported in section 2.4.6 show that under very high densities (where search time is short and coalition time is short), defecting agents are most successful. Several other models of exit strategies bear resemblance to the current study, but have fundamentally different assumptions. Yamagishi et. al. (1994) and Vanberg and Congleton (1992) showed that a cooperative strategy that refrains from interacting with defectors does relatively well. However, they assumed that these strategies had extensive memories for past interactions, while the present model assumes no such memory. Schuessler’s (1989) CONCO strategy was similar to Walk Away except that it immediately entered a new dyad after leaving a partner. Walk Away, on the other hand,
74 must search for a new partner after leaving one. In the real world, there is usually some cost associated with leaving an interaction partner, even if it is just the cost of not interacting for several rounds (until one can find a new partner). When there is no cost associated with finding a new partner (as is the case in Schuessler’s simulations, but not in the simulations reported in this chapter), the conditions are more favorable for a deserting strategy, so its success should be less surprising.
2.5.4. Simple Strategies Revisited The work reported in this chapter shows that not only can contingent movement promote the evolution of cooperation, but that it can do so with minimal cognitive complexity. Some recent work has focused on complex and memory-intensive strategies in the evolution of cooperation, rather than exploring other simple strategies that might promote the evolution of cooperation. Researchers have studied the role of reputation, finding that indirect reciprocity (Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003) and the ability to gossip (Nakamaru & Kawata 2002) increase the likelihood of the evolution of cooperation. Nonetheless, these strategies require both large memory capacities and communication abilities. Other work suggests that costly punishment of cheaters (by first and third parties) can evolve, increasing the viability of the cooperative strategy (Axelrod 1997; Boyd & Richerson 1992). Although punishment seems to be important in human interactions (Fehr & Gachter 2002; Price et al 2002), it requires fairly complex behavioral rules so it is not as likely to have played a role in the first stages of the evolution of cooperation in humans and other species. Subjective commitment has also been explored as a possible
75 means to the evolution of cooperation (Frank 1988; Nesse 2001), but this again requires complex cognitive capacities. As is apparent from the state transition figures (Figures 2.2 and 2.3), Walk Away requires only two states, moving and staying, while TFT and PAVLOV require three (the two-state stationary versions of these fared poorly). In this respect, Walk Away is even simpler than TFT and PAVLOV. Recent research suggests that a number of ‘simple heuristics’ can work better than complex, information intensive strategies (Gigerenzer et al 1999). These strategies often succeed because they exploit the information structure of the environment directly rather than storing all the relevant information in order to perform computations on it. The Walk Away strategy outperforms more complex strategies because it does just that - exploiting the structure of the environment by staying on patches with cooperators and moving away from defectors. Given that a strategy based on movement rules does not require any memory, it can also be used by the simplest of organisms, including those without brains. It could, for instance, be implemented by responses to chemical gradients that are products of different types of interactions. If we assume that organisms originally used simple decision rules and small (or non-existent) memory capacities when making decisions about cooperation, the Walk Away strategy might be a likely candidate because it is both simple and successful under a variety of parameter values. Indeed, Walk Away requires neither the notion of intentionality nor the ability to represent conspecifics. An organism using the Walk Away strategy can respond to others just like it would respond to the inanimate environment: by moving away if it is incurring a cost.
76 Humans might also use something akin to the Walk Away strategy in their social lives. Instead of persisting in a mutually defecting relationship, individuals often seek out new social partners. Previous research has suggested that the tit-for-tat and PAVLOV strategies might approximate human behavior relatively well (Milinski & Wedekind 1998), that people tend to make use of exit strategies when given the option to desert a partner (Yamagishi & Hayashi 1996), and, intriguingly, that the option to exit increases cooperation in individuals who tend to be cooperative in the first place (Boone & Macy 1999; Hauk 1999; Orbell & Dawes 1993). If humans do indeed ‘walk away’ from uncooperative partners - and this increases cooperation among individuals with cooperative tendencies - then this could have important implications for the role of contingent movement in the evolution of cooperation in humans. Although bringing these two findings together in this way is speculative, the combined effect suggests the following: behavioral assortment resulting from contingent movement might increase between group variation in fitness (because those with cooperative tendencies are more cooperative when they have the option to exit), resulting in stronger selection pressures for both cooperation and contingent movement.
2.6. CONCLUSION In sum, the results of the simulations reported here show that cooperative agents with simple contingent movement rules can outperform TFT and PAVLOV under a wide array of parameter values. The Walk Away strategy avoids repeated interactions with defectors and reaps the benefits of interacting with cooperators without compromising its
77 simplicity. These simulations demonstrate that Walk Away is successful when movement cost, error rates and number of patches per agent (inverse density) are low, although extremely low inverse densities favor defectors. Furthermore, the present study shows that Walk Away is an ESS, resistant to invasion by many other strategies, and that Walk Away can invade populations of defectors at lower initial frequencies than either TFT or PAVLOV. Both the adaptationist approach (Williams 1966) and the ‘simple heuristics’ approach (Gigerenzer et al 1999) emphasize the importance of an organism’s environment in understanding its cognitive and behavioral adaptations. The simulation results reported in this chapter illustrate the point that it is often important to model aspects of the social environment, such as spatiality and mobility. The Walk Away strategy would have been impossible to implement in an environment that lacked these features. Indeed, it is these very features - spatiality and mobility – which allowed Walk Away to avoid repeated interactions with defectors and maintain interactions with other cooperators without employing complex strategies.
78
CHAPTER 3 WALKING AWAY FROM THE HAYSTACK: HIGH RATES OF SYSTEMATIC MIGRATION CAN FAVOR THE EVOLUTION OF COOPERATION IN GROUPS
3.0. ABSTRACT Traditional models of the evolution of cooperation through multilevel selection conclude that high rates of migration are prohibitive for the evolution of cooperation. Analytical models typically treat migration rate as a population level variable that acts on individuals with an equal probability regardless of individual and group characteristics. In contrast, the present work uses agent-based techniques to implement systematic migration based on individual level decision rules for movement based on group composition. Agents have two parameters, one specifying whether they do or do not cooperate with group members by investing in a public good and another specifying the level of group cooperation necessary for the individual to stay in the group (the Walk Away rule). Because this Walk Away rule promotes assortment through systematic migration, high levels of cooperation can be selected for even when migration rates are initially high. Here high migration rates are the population level manifestation of individual decision rules to leave insufficiently cooperative groups. Analytical models have shown that random migration can promote the evolution of cooperation in a narrow
79 band of migration rates which can maintain an optimally subdivided population. In contrast, systematic migration via the Walk Away rule promotes assortment in highly mixed populations and also helps to maintain subdivided populations, resulting in selection for cooperation under a variety of migration rates. These findings have potentially important implications for understanding feedback between evolutionary dynamics and population structure in the selection for cooperation in groups.
3.1. INTRODUCTION A number or solutions have been proposed to the apparent paradox of the evolution of cooperation. Kin and reciprocity models have demonstrated that cooperation or altruism (the provision of benefits for others at a cost to the self) can evolve when the costly act provides long-term benefits to the self (Axelrod 1984; Axelrod & Hamilton 1981; Trivers 1971) or benefits to copies of the genes coding for that behavior (Hamilton 1964a; b). More controversially, models of competition between groups (group selection or multilevel selection models) have been proposed as partial solutions to this paradox, demonstrating that increases in cooperation can occur in a population when it is subdivided into groups with varying levels of cooperation (Maynard Smith 1964; Wilson 1987; Wright 1931). A central conclusion drawn from traditional analytical models of multilevel selection is that cooperation or altruism is favored only under intermediate rates of migration between groups in a subdivided population. For instance, Maynard Smith’s classic haystack model (1964) considers a theoretical population of mice distributed
80 among haystacks. These mice interact only with the other mice in their group, leading to higher rates of growth in groups with more altruists. Eventually, these mice leave the current haystack and disperse in order to colonize new haystacks. Population subdivision with some migration between groups is also the essence of island models of the evolution of cooperation (Wright 1931). When rates of migration between groups are high, population subdivision is undermined and between-group selection becomes too weak to favor cooperation. On the other hand, when migration rates are low, defectors simply take over each group and cooperators have no opportunities to create new groups or migrate into other groups. In these situations of low or no migration, the within-group selection component favors defectors. In order for selection at the group level to be strong enough to favor cooperation, a certain amount of mixing is required. However, very high migration rates result in a population of individuals that are completely intermixed, leading defectors to be favored. Only intermediate rates of random migration can favor cooperation in such analytical models. In models of selection among groups, assortment of cooperators within groups makes the evolution of cooperation more likely (Wilson 1983; 1987). However, migration between otherwise subdivided populations can either undermine this assortment or promote it, depending on the rate of migration and depending on whether or migration is random or systematic. Much previous work has investigated the role of different rates of migration on assortment and the evolution of cooperation, but systematic migration has been explored only in a few models (Avilés 2002; Pepper & Smuts 1999; Pepper & Smuts 2002). It has also been shown that periodic group
81 reformation events, in which all individuals are randomly redistributed to groups, can favor altruism when they occur as defectors are just beginning to invade altruistic groups (Fletcher & Zwick 2004; Fletcher & Zwick 2007). The Walk Away rule presented here can promote the effective timing of these group reformation events because individuals begin to leave groups as the level of cooperation in the present group dips below the threshold. Rather than varying the rate of random migration or group reformation events parametrically, as analytical models have done, the present model explores the effects of different movement rules on emigration from groups, suggesting a potentially important role of non-random migration in the evolution of cooperation in groups. In particular, it is shown that agent movement rules for leaving insufficiently cooperative groups can systematically affect population structure in ways that favor the evolution of cooperation. When agents are able to ‘Walk Away’ from ‘haystacks’ with fewer cooperators, groups with more cooperators are more stable and productive, leading to an increase in cooperation in the population. Further, the Walk Away rule can lead to the emergence of dynamic population structures that result in between-group selection for cooperators. In particular, the Walk Away rule can generate a population with a large number of groups, each of relatively small size. Feedback between evolutionary dynamics and population structure can maintain a highly subdivided population structure over time, which favors cooperation. In this way, the Walk Away rule promotes the evolution of cooperation under less restrictive parameter values than have been thought to be necessary (based on the results of analytical models).
82 Throughout this chapter, the term ‘cooperation’ is used to describe the groupbeneficial behavior of the agents in this model. The terms cooperation and altruism are often used interchangeably to refer to behaviors that benefit others at a cost to the self, but cooperation can also refer to a broader class of behaviors that benefit others such as those underlying mutualisms, interactions between kin and reciprocal interactions. Such interactions are not typically considered altruism (unless qualified as ‘kin-based altruism’ or ‘reciprocal altruism’) because they benefit the individual (or the individual’s genes) in the long term. The long-term benefits associated with the use of the Walk Away strategy for a particular individual are not systematically explored here, so the group-benefiting behavior is referred to as cooperation rather than altruism.
3.1.1. Walk Away The Walk Away rule is an individual level decision rule for leaving partners or groups that are insufficiently cooperative. The importance of partner choice or exit in the evolution of cooperation has received attention in the past decade and half. Theorists have pointed out that partner choice can play a role in promoting cooperation both between species (Bergmuller et al 2007a; Bull & Rice 1991; Connor 1992; Friedman & Hammerstein 1991; Noe & Hammerstein 1994; Sachs et al 2004) and among members of the same species (Bergmuller et al 2007a; Sachs et al 2004). Experimentalists have explored partner choice in experimental economic games with humans (Barclay & Willer 2007; Boone & Macy 1999; Orbell et al 1984). Modelers have examined partner choice decision rules of varying complexities, from simple rules that respond only to the most recent action of a partner (Aktipis 2004) or the state of the local environment (Pepper &
83 Smuts 1999; Pepper & Smuts 2002), to more complex rules that remember past behavior of partners (Vanberg & Congleton 1992; Yamagishi et al 1994) and allow for contingent cooperation as well as contingent movement (Hamilton & Taborsky 2005). It has been noted that spatial mobility can constrain the evolution of cooperation when there is low search times for new partners (Enquist & Leimar 1993), and when only defectors can use movement to exploit regions of cooperators (Dugatkin & Wilson 1992). The present study is an extension of the dyadic Walk Away model (Aktipis 2004). In the dyadic model, agents were placed on a lattice and interacted with a partner on the same cell in a prisoner’s dilemma game. Walk Away agents were able to move away from partners who defected, outperforming Tit-for-Tat under a variety of parameter values. Here, the Walk Away strategy is generalized to group-wise interactions, allowing for groups of various sizes and endogenously determined migration rates, factors rarely included in models (but see Avilés 2002; Pepper 2007). The present simulations investigate the effects of different Walk Away thresholds on migration rates and the feedback between population composition and spatial dynamics. The ability to Walk Away from uncooperative groups enables individuals to ‘choose’ whether to stay in a group. However, existing groups are not endowed with the ability to ‘choose’ who they will or will not allow to enter into the group (although individual members of the group can choose to leave if the introduction of a new member reduces the level of cooperation in the group). Recent work has shown that a probabilistic movement rule very similar to the Walk Away rule promotes positive assortment, i.e., the preferential association of individuals of the same type, when used by cooperators (Pepper 2007). This work demonstrated the
84 important role that systematic movement away from less cooperative groups can have on population structure. Because systematic migration can promote assortment, cooperation can be favored in the present simulation, even when migration rates are high.
Walk Away
Figure 3.1. The Walk Away strategy is illustrated in the above state transition figure. Agents can occupy one of two states, “move” and “stay,” indicated by boxes; arrows indicate possible state transitions. Agents stay in a group if and only if the return (R) received from the group meets or exceeds the agent’s threshold (T). Cooperators in the “stay” state contribute to the group each time period and stay on the patch. Defectors in the “stay” state simply remain in the group, contributing nothing each time period. In the “move” state, agents move one step each time period and do not contribute. Agents switch to the “stay” state when they encounter another agent (or agents), staying only if the benefit received from the group exceeds their threshold.
3.1.2. Social Dilemmas and Multilevel Selection In the present simulations, agents interact with others in their group in a social dilemma (Dawes 1980) or public goods game (Ledyard 1995). In a social dilemma, individuals have an incentive not to cooperate with the group, but every individual in the group would be better off if every other individual cooperated. In these group-wise interactions, individuals can behave selfishly, keeping benefits for themselves, or altruistically, bearing a cost in order to benefit other group members. As with standard Prisoner’s Dilemma (PD) games, in one-shot interactions, the individually optimal
85 behavior is to free ride, contributing nothing to the group (defect). In contrast, the group-wide optimum is for each individual to contribute as much as possible to the public good (cooperate). However, when individuals engage in repeated interactions, the analysis of the individually optimal behavior is significantly more complicated, and it is often not pure defection (Axelrod 1984). The fundamental principles of multi-level selection can be understood in the context of a social dilemma (Fletcher & Zwick 2007): within a group of individuals who exert fitness effects on one another, selection will favor those who do not deliver benefits to the group at a cost to themselves because these individuals will enjoy higher withingroup fitness. However, groups that consist of individuals who provide fitness benefits within the group can increase in size more quickly than groups with fewer such individuals. If this between-group selection is stronger than within-group selection, the proportion of altruists in the overall population can increase while the proportion of altruists in each group declines, an effect known as Simpson’s Paradox (Simpson 1951; Sober & Wilson 1998). The notion that selection can act at the level of the group is still controversial, despite the increasing acceptance of the idea that the level at which selection acts can vary in different contexts and demonstrations that multilevel selection and kin selection are mathematically equivalent, topics reviewed elsewhere (Queller 1992; Wilson & Wilson 2007). Traditional models of multilevel selection typically assume migration as an exogenous variable (Maynard Smith 1964; Wright 1931) as do a number of recent models of the evolution of cooperation in groups (Janssen & Goldstone 2006; Killingback et al 2006; Traulsen & Nowak 2006). Models of the evolution of
86 cooperation that parametrically vary group size and composition have shown that selection favors more cooperative individuals when groups are small (Boyd & Richerson 1988; Nowak et al 2004), the benefits to others are large (Fletcher & Zwick 2004), and the proportion of cooperators in each group varies (Fletcher & Zwick 2001). Agent based methods provide an alternative to parametrically varying migration rate, group size or number of groups parametrically, instead allowing them to emerge from the behavior of individuals. In this way, agent based simulations allow these features to be generate from the bottom up, allowing for the emergence of novel and sometimes unexpected phenomena. Agent based models of cooperation that allow migration, group size and/or number of groups to spontaneously emerge include the dyadic Walk Away model (Aktipis 2004), models of the effect of environmental feedback on movement (Pepper & Smuts 1999; Pepper & Smuts 2002), models of group joining (Aviles 1998; Avilés 2002) and Pepper’s (2007) non-evolutionary simulations in which the probability of movement/migration between patches was proportional to the number of defectors in that patch. In the present agent based simulations, individuals use a Walk Away rule, moving away from insufficiently cooperative groups (moving if the return from the group is below the Walk Away threshold). This results in systematic migration, group dissolution and group formation, and selection for cooperation under a variety of parameter values In these simulations, agents use the Walk Away rule, which specifies the level of benefits (which can also be conceptualized as the proportion cooperators) the agent requires to stay in the current group. Both cooperators and defectors use this same rule, but the level of benefits required to stay in the current group (the leaving threshold) can
87 vary between cooperators and defectors. This is in contrast to analytical models which do not allow for systematic migration, and different from those that do (e.g., Avilés 2002; Pepper 2007), in that the threshold is parametrically varied here. The most novel component of the present work is the demonstrated capacity of Walk Away to sustain dynamic population structures that promote the evolution of cooperation.
3.2. MODEL RESULTS An agent based simulation was used to implement this group-wise cooperation simulation. Unless otherwise noted, simulations start with 500 agents, 5% of which are cooperators and 95%, defectors. A typical simulation takes between 20,000 and 40,000 time periods for cooperators to increase in frequency and plateau (if they do so at all, see final section of results for details), although cyclical variations around the maximum exist for some parameter values. Results are reported at 50,000 time periods unless otherwise noted. Because the primary issue of interests is the responsive movement rule, the parameters underlying this rule were parametrically varied. Methodological details can be found at the end of this chapter. The first section of the results reports the outcomes of parametrically varying the leaving threshold of cooperators and defectors on the proportion of cooperators, including the addition of noise into the Walk Away threshold. The second section reports the migration rate at various leaving thresholds with and without evolutionary dynamics. The final section examines the role of Walk Away in creating and maintaining
88 subdivided population structures and the feedback between structural and evolutionary dynamics in maintaining between-group selection for cooperators.
3.2.1. Cooperator Viability Cooperators were most successful when the thresholds of either or both types of agents were high. When cooperators had a high threshold, they were able to leave groups with growing numbers of defectors. Interestingly, when defectors had a high threshold, this also promoted the success of cooperators because defectors did not spend long periods of time in groups where they could benefit from exploiting cooperators. Defectors with high thresholds left groups, after which they were less likely (than cooperators) to be able to form successful and stable groups. Alternatively, if defectors had thresholds of 0, they would never leave a group they had joined, even if it had been completely overtaken by defectors. In other words, with thresholds of 0, defectors had no mechanism by which to colonize new groups. This enabled cooperators to be relatively successful when defector threshold was 0 and there was no noise (see Figure 3.2). However, this effect disappeared when some noise was introduced to the Walk Away threshold (Figure 3.3). These basic finding are illustrated in Figures 3.2 and 3.3, below, which show the results as a 2-dimensional surface where the y-axis represents the proportion of cooperators at various combinations of cooperator and defector thresholds. With no noise (Figure 3.2), a mixed population of cooperators and defectors was maintained over the majority of the parameter space. Defectors dominated the population when the
89 thresholds of both cooperators and defectors were low (but not when defector threshold was 0).
Figure 3.2. This figure shows the percent of cooperators at various cooperator and defector thresholds at 50,000 time periods. When the threshold of cooperators or the threshold of defectors was higher, cooperators were more successful. When the threshold of both strategies was low, defectors outperformed cooperators (except when defector’s threshold was 0).
Variation was introduced into the movement threshold of agents, such that the movement threshold used by a given agent varied between time periods around a mean (reported on the graph) with a standard deviation of .1. This made the likelihood that an agent would stay in a group probabilistic around the reported mean. This resulted in smaller regions with a mixed population and larger areas with only cooperators and only defectors. In addition to the steeper slope between these regions, the moderate success of cooperators when the average initial defectors threshold was 0 disappears (see Figure
90 3.3a). This is because defectors probabilistically used a threshold higher than 0, giving them way to leave previously colonized groups and take over new ones. a)
b)
Figure 3.3. These figures show the percent of cooperators at various cooperator and defector thresholds at 50,000 time periods. a) When variability (with a SD of .1) was introduced into the threshold used by the agents, the regions of cooperator success at high thresholds expands. b) When variability was increased to SD = .3, this region is even larger and the slope between the regions of cooperator and defector success was steeper.
91 When the standard deviation was increased from .1, the slope was steeper and the region favoring cooperators grew, and the opposite was the case when the standard deviation was decreased to .05. When the standard deviation of the variation was raised to .3, cooperators were overwhelmingly successful, but a mixed population or complete dominance of defectors is still apparent when the threshold of both cooperators and defectors was lower than .5 (Figure 3.3b).
3.2.2. Migration In this section, migration rate is reported as a function of the thresholds of both cooperators and defectors. Because there can be high variability between time periods in migration events, the average rate of migration over the final 100 time periods of runs is used, and migration is calculated as the rate of emigration from groups (the number of agents leaving divided by the number agents leaving and staying). In these simulations, there was a noise level of SD = .1 (compare to Figure 3.3a) in the agents’ thresholds and the initial proportion of cooperators was .05. First, non-evolutionary results are reported showing the migration rates that emerge from parametric variation of the cooperator and defector Walk Away thresholds with population compositions consisting of different proportions of cooperators. Second, results from evolutionary simulations show that the final migration rates in a population of agents can differ dramatically from the initial migration rates before selection.
92 3.2.2.1. Migration dynamics in the absence of selection In order to investigate the population dynamics that result from agent behavior in the absence of evolutionary dynamics, a set of simulations without reproduction and death were run for 1,000 time periods (long enough for the migration rate to reach equilibrium) to provide a baseline for comparing the approximate initial migration rate in evolutionary simulations with the long range evolutionary outcome. The average migration rate over the final 100 time periods is reported in Figure 3.4a-c below. Note that the lowest migration rates emerged when agents had low thresholds, essentially being more tolerant of low levels of cooperation. Very high levels of migration resulted when agents, especially cooperators, had high thresholds. In other words, intolerance of low levels of cooperation leads to more group leaving and therefore higher rates of emigration from groups. Very high migration rates result when the initial proportion of cooperators is .05 (the value used in all other simulations reported here unless otherwise noted). When the proportion of cooperators is increased, the migration rate decreases dramatically, with intermediate values for proportions of .5 (Figure 3.3b) and very low levels for proportions of .95 cooperators (Figure 3.3c).
93 a)
b)
c)
94 Figure 3.4. These results from non-evolutionary runs show the migration rate that emerges from agents’ thresholds with various proportions of cooperators (reported at 1,000 time periods). a) When the proportion of cooperators is .05, high migration rates result overall, but especially when cooperator and/or defector thresholds are high. b) When the proportion of cooperators is raised to .5, migration rates are lower. c) When the population consists of .95 cooperators, the migration rate is very low.
Interestingly, it is exactly the regions with high initial migration rates in Figure 3.4a (70 – 99%) in which the evolution of cooperation was favored (compare to Figure 3.3a), seeming to contradict the results of traditional analytical models which show that high migration does not allow cooperation to evolve (Maynard Smith 1964; Wright 1945). Rather than being a barrier to the evolution of cooperation, the high migration rate can be considered a population level manifestation of high Walk Away thresholds, which promote assortment and therefore the evolution of cooperation. It is exactly the high thresholds of these agents that result in a high migration rate and enable cooperation to be selected in the long term (Figures 3.2, 3.3). 3.2.2.2. Migration dynamics with selection In the absence of evolutionary dynamics, agents with higher Walk Away thresholds created population dynamics characterized by high rates of migration. However, over evolutionary time, the differential stability and growth of groups made up of cooperators lead to increases in the number of cooperators, which then influenced the rate at which agents Walk Away thresholds are reached, lowering the migration rate. Figure 3.4a-c demonstrates this, showing that increasing the proportion of cooperators decreases the migration rate in the population.
95 Evolutionary results suggest that, over time, systematic differences in the stability and viability of groups of cooperators lead to higher payoffs and the subsequent evolutionary success of cooperators. Further, the rate of migration at the end of evolutionary simulations was very different from the rate of migration that emerges in the non-evolutionary simulations. In fact, the regions with higher initial rates of migration (for initial proportions of .05 cooperators, Figure 3.4a) had lower final rates of migration in the evolutionary simulations because cooperators were actually selected over the region where migration rates were high (Figures 3.3a, 3.5). The high migration rate is a manifestation of the high Walk Away thresholds being used by the agents, and high thresholds for favor cooperators (Figures 3.2, 3.3). The final migration rates in the evolutionary simulations (Figure 3.5) can be decomposed into two basic components, one where defectors dominate at low thresholds of both types (which contains a component similar to Figure 3.4a for low thresholds of both) and another where cooperators dominate for high thresholds of either type (containing a component similar to Figure 3.4c for high thresholds).
96 Figure 3.5. In evolutionary simulations, the pattern of migration rate is the opposite to that which is the case before the action of selection (compare to Figure 3.3a which reports non-evolutionary results for 5% cooperators). Final migration rate (at 50,000 time periods) is highest when agents have low thresholds. When the thresholds of either cooperators or defectors are high, relatively low rates of migration result.
3.2.3. Dynamic population structures Structural dynamics, which emerge from individual movement rules, and evolutionary dynamics, which change the proportion of cooperators, can reciprocally influence one another, resulting in surprising and potentially counterintuitive effects such selection for cooperation under high initial migration rates. The quantitative and qualitative findings presented below suggest that dynamic population structures may play an important role in the evolution of cooperation in this model. 3.2.3.1. Dynamics in a single run Figure 3.6 shows the changes in a number of variables over time in a single simulation. The results below are reported from a single run in which both cooperators and defectors have a threshold of .7 (with a .1 SD for noise). In contrast to simulations reported earlier, if all defectors went extinct, a single cooperator occasionally mutated into a defector (see methods). This allowed for the examination of the dynamics underlying the maintenance of cooperation in the face of invasion by defectors. The results reported in this section describe the changes in population structure and dynamics as cooperators are selected. At the beginning of the simulation, agents were randomly distributed on the grid. Agents then moved, leaving uncooperative groups and staying in cooperative ones,
97 resulting in high rates of migration since the population is initially made up of mostly defectors. However, groups made up of cooperators were more stable than groups containing defectors, allowing for cooperation to slowly increase (blue line), which decreased the rate of migration (black line), increased the number of groups (green line) and average group size (yellow line). Once the population reaches high levels of cooperation, the group size remains high, indicating high population subdivision. This high subdivision helps to maintain selection for cooperation through the feedback loop described in the next section. % Coop # Groups Group Size Mig. Rate
Population Dynamics and Selection for Cooperators 140
7
120
6
100
5
80
4 Group
60
3
40
2
20
1
0
Size
0 0
20000
40000
60000
80000
100000
Time
Figure 3.6. As the proportion of cooperators increases (blue line), the migration rate (black line) decreases, resulting in a larger number of groups (green line) and larger group size (yellow line). Once cooperation has increased, the group number and group size remain high and the migration rate stays relatively low. This plot shows the changes in each variable over the first 100,000 time periods of a run with cooperator and defector thresholds of .7. Percent cooperators, migration rate and number of groups are indicated on the left axis and group size is indicated on the right axis.
98 The changes in these variables over time suggest a number of interesting conclusions. Firstly, the migration rate was very high during the times at which selection favored cooperators. In fact, the migration rate was close to 1 at the beginning, meaning almost all agents immediately left groups that they joined. The migration rate reached a low level (varying between about 0-5%) only after the proportion of cooperators had increased dramatically. If it were migration rate, per se, that limited the evolution of cooperation, it would be impossible to explain this pattern. However, the migration in this simulation was systematic rather than random, and non-random migration (as accomplished here through responsive movement away from less cooperative groups) can favor the evolution of cooperation because it promotes assortment (Pepper 2007). The results presented above combined with previous work suggest that the rate of migration itself is less important than the reasons for that migration. To the extent that migration occurs as a result of responsive movement away from uncooperative groups, high levels apparently do not restrict the evolution of cooperation. 3.2.3.2. Group Stability Agents use the Walk Away rule of leaving groups with insufficient returns (Figure 3.7a). This results in greater stability of groups made up of more cooperators and lower stability of groups made up of more defectors (Figure 3.7b). Higher rates of migration emerge in the population when the proportion of cooperators is lower because more agents leave groups (Figure 3.7c). However, it is not the migration rate itself which is important in the evolution of cooperation in this model, it is the fact that emigration is more likely to occur from groups with lower levels of cooperation. The lower stability of
99 less cooperative groups generates population-level dynamics that can favor the evolution and maintenance of cooperation.
Figure 3.7. a) The individual level Walk Away rule specifies the level of return required by an agent to stay in the current groups. b) This results in great stability of groups with more cooperators and lower stability of groups with more defectors. c) Higher rates of migration results when cooperation is lower and lower levels of migration result when cooperation is higher.
The schematic presented below (Figure 3.8) represents the relationship between population subdivision (group number) and stability of the population (migration rate) in the present model. As stability of the population increases (migration decreases), the population first becomes more subdivided as loners begin to join and stay in groups. This continues until further increases in stability decrease group formation events, resulting in the long-term outcome of all individuals being part of the single most successful group.
100 This general relationship between population subdivision and population stability holds for analytical models of the evolution of cooperation as well. Indeed, intermediate rates of migration (intermediate population stability) between groups favor the evolution of cooperation because the resultant population subdivision favors cooperation via between-group selection.
Figure 3.8. This schematic shows the relationship between population stability and population subdivision. As the stability of the population structure increases, loners/migrants are more likely to join and stay in groups, increasing the number of groups. After a certain point, however, further increases in population stability result in fewer groups as the population structure moves towards being a single group with all individuals. When there are more groups, cooperators are favored via between-group selection (y-axis). Because agents use a Walk Away rule of leaving insufficiently cooperative groups, increases in the proportion of cooperators causes greater structural stability. This results in a positive feedback loop (blue arrow) that can favor cooperation at low levels of structural stability (high migration rates) and a negative feedback loop (red arrow) that reduces structural stability once it has increased past the peak.
101 When migration rate is varied parametrically in analytical models, the structural stability of the population is fixed over the course of the simulation. This leads to the conclusion that only intermediate migration rates (with result in sufficient population subdivision) can favor the evolution of cooperation (Figure 3.9a). In contrast, the Walk Away model allows the stability (migration rate) to emerge from the behavior of the agents. This creates a feedback loop wherein the proportion of cooperators within groups and in the overall population influences the migration rate and group stability (Figure 3.9b). Because the relationship between the stability of the population and population subdivision is positive before the peak and negative after the peak (Figure 3.8), this feedback loop can both promote population subdivision (move it towards the peak) and maintain the population in a highly subdivided state (maintain it at the peak). More specifically, the positive relationship between population stability and population subdivision before the peak results in a positive feedback loop wherein population stability and differential group stability increases the frequency of cooperators and the increase in cooperators further increases stability. In contrast, after the peak is passed, further increases in population stability decrease the subdivision of the population (promoting fewer and larger groups), resulting in greater selection for defectors. This results in a negative feedback loop where increased stability decreases the frequency of cooperators, which then decreases stability. The positive feedback loop allows cooperation to be selected even at initially high rates of migration (low stability), and the negative feedback loop maintains the population at the peak of population subdivision, resulting in continued between-group selection for cooperators.
102
Figure 3.9. Population stability increases population subdivision up to a certain point, after which it decreases population subdivision (yellow arrow, see also Figure 3.8). Higher population subdivision promotes between-group selection (blue arrow), which favors cooperators (blue arrow). a) In analytical models migration rate (stability) is parametrically varied, resulting in the conclusion that cooperators are favored only at intermediate rates of migration. b) In contrast, when agents use the Walk Away rule, this creates feedback between the proportion of cooperators and stability (blue arrow outside box). Positive feedback (when the relationship between stability and population subdivision is positive) allows cooperators to be selected at initially high levels of migration (low stability) and negative feedback (when the relationship between stability and population subdivision is negative) maintains population subdivision at a high level, enabling continued between-group selection for cooperation. Further, the Walk Away rule
103 results in groups made up of cooperators being more stable, which promotes assortment (blue arrow) and between-group selection for cooperators (blue arrows).
3.3. DISCUSSION Despite analytical findings that cooperation cannot be favored when migration rates are high, the present agent-based model demonstrates that cooperators can actually be favored at high migration rates. This counterintuitive result occurs because systematic migration from less cooperative groups can increase positive assortment (Pepper 2007) rather than decreasing it as random migration does. As assortment increases, between group selection is more effective at selecting cooperators. The dynamics resulting from agent behavior in this simulation demonstrate that the capacity for responsive movement causes non-random migration, which can increase assortment and population subdivision, promoting the strength and effectiveness of between-group selection. In other words, individual level movement rules can influence the way selection acts on groups. High rates of migration in this simulation are a manifestation of high Walk Away thresholds, which can lead to the evolutionary success of cooperators (Figures 3.2 and 3.3). Because agents used the Walk Away rule, leaving insufficiently cooperative groups, migration rate was higher when there were more defectors and when Walk Away thresholds were high (as demonstrated in Figures 3.4 and 3.5). The model presented in the final section (Figures 3.8 and 3.9) suggest that the Walk Away rule establishes a feedback loop between the proportion of cooperators and stability which both selects for cooperation and helps to maintain it.
104 In Fletcher and Zwick (2007), the time between group reformation events was varied parametrically and they found that cooperation could be favored if group reformation occurred as defectors began to be selected. The Walk Away rule might serve as a decentralized mechanism that can promote such regrouping events when the between group selection component (favoring cooperators) would otherwise be outweighed by selection within groups (favoring defectors). In other words, the Walk Away rule can provide a way for group reformation events to occur when defectors might otherwise invade.
3.3.1. Group and Multilevel Selection There has been a considerable degree of confusion surrounding the relationship between multilevel selection and kin selection despite theoretical and mathematical demonstrations that kin selection is a subcategory of multilevel selection (Queller 1992; Wilson 1983; 1987). Kin selection and group selection both rely on the differential partitioning of genetic variance between versus within groups such that individuals in the same group have a higher probability of sharing identical genes for the trait being selected. In the case of kin selection, this is due to decent, while multilevel selection does not distinguish between genetic identity due to decent as opposed to other causes of assortment. Even simple multilevel selection models rely on the differential growth of more cooperative groups, and this necessarily implies that offspring stay in their parent groups for some minimal time before regrouping occurs. Some effects in multilevel selection models can be traced to effects that kin have on each other and the agent-based nature of this model makes this component easier to see.
105 However, in the present model, assortment is not only a result of local reproduction and relatedness, it is also the result of active assortative processes. Because individuals use responsive movement away from insufficiently cooperative groups, groups made up of individuals who are highly cooperative are more stable and longlasting than less cooperative groups. In Pepper and Smut’s (2002) model of assortment through environmental feedback, high consumers (defectors) and low consumers (cooperators) move between patches that have plants with a logistical growth rate. Agents move towards patches with higher energy levels, leaving areas that have been exploited. Pepper and Smuts show that this foraging rule resulted in positive assortment through the differential effects agents had on their environments. In Pepper and Smuts’ (2002) model, regions with more cooperators tended to be more stable. Likewise, in the dyadic Walk Away model (Aktipis 2004), dyads made up of two cooperators were stable while those made up of even one defector did not last more than one time period. A similar effect is at work in this group Walk Away model, since individuals who are in groups with more cooperators are more likely to stay because the level of cooperation stays above the Walk Away threshold. These simulations all demonstrate the fundamental concept that regions of cooperators can be more stable than regions of defectors, leading to greater opportunities for cooperators to capture benefits. Recent work by Pepper (2007) has demonstrated that movement away from less cooperative groups can change assortment. In particular, the probabilistic responsive rule modeled by Pepper shows that responsive movement, when used by cooperators, promotes positive assortment. The dynamics underlying cooperator success in the present
106 simulations rely on the fact that the Walk Away rule promotes systematic migration and can increase assortment. The present model differs, however, in that varying movement threshold were explored and the effects of the Walk Away rule on population structure and selection dynamics are explored here. 3.3.2. Individuals Rules Affect Population Dynamics Responsive movement is simple, presumably easy to evolve, and could have played an important role in the evolution of cooperation. To the extent that systematic migration changes population dynamics, the direction of selection on traits that effect group members (as cooperation does), can be dramatically influenced. This suggests that, in the natural world, there may be important interactions between levels that have previously been considered separately, and might not be easily analytically joinable. When individual level behavior promotes changes in spatial dynamics, agent based simulations might be the most effective computational tool for addressing these interactions (Hammond & Axelrod 2006). The results of the simulations reported here demonstrate that a simple Walk Away strategy can promote the evolution of cooperation in groups. This is the result of nonrandom migration, which leads to greater stability and evolutionary success and of more cooperative groups. Perhaps even more importantly, the present simulations suggest that systematic emigration via the Walk Away strategy can favor subdivided population structures that make the evolution of cooperation more likely, allowing for the evolution of cooperation, even at high rates of migration. If organisms in the natural world migrate because of decision rules such as the Walk Away rule or more complex rules such as
107 those underlying optimal foraging (MacArthur & Pianka 1966), the assumption of random migration used in analytical models might lead to incorrect conclusions about the viability of cooperation. The simplicity of the Walk Away rule suggests that even organisms with very limited information processing capacities could use this rule. Why, then, is cooperation not more common in the natural world? There are several potential answers to this question. First, it might be the case that cooperation is actually much more common than has previously been thought. Indeed, the interactions among cells and their components (Maynard Smith & Szathmary 1995) as well as ‘social’ interactions among amoebas (Strassmann & Queller 2007) have been characterized in terms of cooperation and multilevel selection, suggesting that cooperation between very simple entities is not only possible, but potentially widespread. Second, it is possible that social organization in the natural world can be constrained by factors that do not allow for assortment from individual Walk Away rules. For example, the ability of defectors to preferentially track and exploit cooperators could limit assortment even if cooperators were using a Walk Away rule. Third, there might be physical or ecological constraints that create high levels of mixing or highly stagnant population structures, making it impossible for the Walk Away rule to promote population subdivision in a way that would favor cooperation. A fourth possibility is that defectors might often evolve to have a threshold which enables them take advantage of the regions in which they will outperform cooperators. If cooperators’ thresholds are significantly less than 1, defectors with very low thresholds can dominate the population. The speed and effectiveness of selection for cooperators with higher thresholds and defectors with lower thresholds is a topic for
108 further investigation, and one the might shed some light on the question of when and why we should expect to see cooperation in nature. Preliminary work has shown that the speed of selection on defectors’ thresholds is more rapid than selection on cooperators’ thresholds, suggesting that defectors might have an advantage over cooperators by virtue of the quicker evolution of their thresholds to levels that allow for exploitation of cooperators. It is shown here that a very simple rule of ‘Walking Away’ from insufficiently cooperative groups can promote the evolution of cooperation. Further, the Walk Away rule provides an alternative to the assumption of random migration inherent in haystack and island models. It is proposed that the Walk Away rule closes the feedback loop between cooperator frequency and population stability (migration) that has been left open in analytical models. The simple Walk Away rule also promotes systematic migration, which promotes assortment and maintains population subdivision which both increase selection for cooperation. In order to investigate the causes and effects of systematic migration, we have to be willing to ‘Walk Away’ from the haystack, leaving the idea that migration rate is a population level variable. By moving towards a conception of population stability as a manifestation of potentially diverse individual decision rules operating in varied local environments, it becomes possible to investigate the subtle interactions among movement rules, spatial structure and evolutionary dynamics in models of the evolution of cooperation.
109
3.4. METHODS Simulations were programmed in NetLogo 3.1.2 (Wilensky 1999), an agent based programming platform. This platform enables the modeling of social spatial interactions among agents using a decentralized approach of programming the individual movement rules and interaction rules rather than creating equations of population level dynamics. Agents are embedded in a spatial lattice in which they interact with individuals on the same ‘patch.’ All agents use the Walk Away movement rule, staying in groups that have sufficiently high levels of cooperation and leaving groups that to not meet that cooperation threshold. When agents leave, they move forward in the spatial lattice, changing their direction slightly during each step. When two or more agents land on a single cell they constitute a group and play a public goods game during each time period that the group is maintained. Group size is determined by agent movement into and out of groups as well as the reproduction of agents within groups. Agents begin with an energy level between 0-2000 (in a uniform distribution). During a time period each agent has a chance of being eliminated from the population and when an agent is eliminated a random agent is given the opportunity to reproduce. Approximately 2 agents have the opportunity to reproduce during each time period, but they do so only if their energy level is greater than 1000 (making it possible for the population size to decrease over time). Parents split their energy with their offspring who stay in the same group as the parent. In this model, density was 5 patches per agents, and the multiplier was 1.9, allowing for a social dilemma even in groups of two individuals (unlike Killingback et al 2006 in which social dilemmas existed only for sufficiently
110 large groups). Simulations start with 500 total agents, 5% of which were cooperators. No mutation occurred. The behavior of individual agents and the overall performance of the model were validated by examining the changes in individual variables during group-wise interactions to ensure that individuals were interacting according to the payoff framework described earlier. Further, individual agents were ‘followed’ over the course of trial runs to determine whether reproduction and death subroutines were functioning as specified. Where necessary, the aggregate effects on migration rate or cooperator frequency were explored to ascertain whether the outcomes were consistent with analytical results.
111
CHAPTER 4 THE INDIVIDUAL IN GROUP SELECTION: HOW RESPONSIVE MOVEMENT SHAPES THE FORCES OF SELECTION
4.0. ABSTRACT Can cooperation evolve via genetic group selection? This question has been at the forefront of the controversy surrounding group selection for decades (Wilson 1983; Wilson & Wilson 2007), and many have argued that genetic group selection is unlikely to promote the evolution of cooperation in humans (Boorman & Levit 1980; Richerson & Boyd 1998; Williams 1966). The simulations described here demonstrate that very simple individual level decision rules can lead to population changes that can promote multilevel selection, suggesting that the parameters under which group selection can occur are not as rare as some have argued. A Walk Away model of group cooperation is described in which responsive movement causes systematic migration out of less cooperative groups and differential stability of cooperative groups. Because individual decision rules can influence aggregate dynamics in ways that are important for the operation of selection between groups, agent based simulations can provide unique insights into the operation of multilevel selection and the potential feedback between selection at different levels of organization.
112
4.1. INTRODUCTION Traditional approaches to analyzing the effectiveness of genetic group selection as a selective force promoting the evolution of cooperation have made use of analytical models of group selection (Maynard Smith 1964; Wilson 1983; 1987; Wright 1945). Models such as these show that there are a limited set of parameters (e.g., migration rates, temporal constraints) under which group selection can favor the evolution of cooperation, leading some to conclude that genetic group selection must be rare (Williams 1966) and that group selection is unlikely to have played an important role in the evolution of cooperation in humans (Richerson & Boyd 1998; Richerson & Boyd 2005; but see Sober & Wilson 1998). The fundamental argument of this chapter is that analytical models typically do not take into account the potential role of agent-generated behavior in shaping evolutionary dynamics. These models neglect the effects that responsive movement has on positive and negative assortment, migration, and group size. Assortment increases the proportion of interactions that cooperators have with one another, allowing cooperators to avoid a certain amount of exploitation by defectors. When a population is assorted based on cooperative tendencies, cooperators can be much more successful than in a randomly mixed population (Aktipis 2000; 2004; Eshel & Cavalli-Sforza 1982; Pepper & Smuts 2002; Wilson 1983; Wilson & Dugatkin 1997; Wilson et al 1992). Analytical models of cooperation in groups typically assume that individuals are passive and do not respond to their physical and social environments in ways that systematically change assortment.
113 Conclusions generated from such models might not be generally applicable to species capable of responsive movement. Agent based models of the evolution of cooperation have the potential to inform questions about the evolution of cooperation in groups. These models allow the decisionrules of individual agents to be specified in a straightforward manner, revealing the aggregate effects of these rules on assortment and subsequent group selection. This chapter aims to show that the possibility of responsive movement greatly increases the potential for assortment and therefore can promote genetic group selection as a force promoting the evolution of cooperation.
4.1.1. Walk Away The simulation described in this chapter demonstrates that very simple strategies can underlie cooperation in groups. The Walk Away strategy allows for movement that is contingent upon the level of aggregate cooperativeness. The implementation of such a decision rule requires little cognitive complexity, involving only two states (move and stay), two transition rules and two parameters (I, specifying the level of investment in the group and T, specifying the Walk Away threshold of the agent), as illustrated in Figure 4.1. The simplicity and success of the Walk Away strategy in past work (Aktipis 2004; manuscript) suggests a pathway for the evolution of cooperation in groups, even in simple organisms. For example, a single celled organism can act like a Walk Away agent, producing a fixed amount of a given protein (I) and staying only in areas with sufficiently high levels of the protein (R > T).
114 4.1.2. Background The Walk Away strategy differs from earlier work in that the agents in the current simulation are very simple, in contrast to the more complex agents in other simulations (Vanberg & Congleton 1992; Yamagishi et al 1994) and the agents interact on a spatial lattice rather than being paired randomly (Schuessler 1989). In previous work, contingent movement strategies have been implemented in dyadic interactions (Aktipis 2004; Enquist & Leimar 1993; Hamilton & Taborsky 2005; Nowak & Sigmund 1993; Schuessler 1989). The Walk Away strategy has also been modeled in the context of small group interactions where group contributions are binary (Aktipis manuscript). In order to demonstrate in an intuitive manner the effect that the Walk Away rule has on the formation, success and dissolution of groups, the binary Walk Away model is extended to the continuous investment case, which allows for much larger groups and richer dynamics than those that emerge when investment is binary. Further, the agent based method allows for a spatial implementation which makes the simplicity of the responsive movement strategies easier to appreciate and the dynamics underlying the evolution of cooperation more intuitive.
4.1.3. Complexity and Simplicity of Decision Rules Clearly humans have the necessary cognitive capacity to act like “Walk Away” agents, but others have suggested that complex abilities such as indirect reciprocity (Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003; Panchanathan & Boyd 2004), gossip (Nakamaru & Kawata 2002), punishment (Axelrod 1997; Boyd & Richerson 1992; Fehr & Gachter 2002; Price et al 2002) and subjective commitment (Frank 1988;
115 Nesse 2001) are likely to be important. However, the fact that humans use a host of complex cognitive mechanisms when interacting in their social worlds does not necessarily entail that those abilities are necessary for the maintenance or initiation of cooperation. In fact, recent work suggests that examining simple strategies underlying decision-making in general (e.g., Gigerenzer et al 1999), and cooperation in particular (Aktipis 2004; Aktipis 2006), can lead agents to make accurate decisions and/or behave in a way that leads to evolutionary success.
4.2. SEMI-ISOLATED POPULATIONS Standard models of genetic group selection involve semi-isolated populations, such as islands or haystacks, where groups are relatively stable, but there is occasionally mixing or migration between groups (Maynard Smith 1964; Wilson 1983; 1987; Wright 1945). This can lead to group level selection for cooperation. In order to see why this is the case, consider two groups. In group A, 80% of the individuals are cooperators (individuals who benefit the group at a cost to self) and 20% are defectors (individuals who exploit the group without contributing). The opposite is the case for group B, being made up of 20% cooperators and 80% defectors. Within each group, the proportion of defectors would increase after each generation because defectors will always have a higher payoff than cooperators in the same group. However, the group with more cooperators will increase in size more quickly than groups with fewer cooperators (because groups with more cooperators will have a higher average payoff per individual). This can lead to an increase in the number of cooperators in the population overall, even
116 though defectors have the advantage within any particular group (Sober & Wilson 1998). This model requires intermediate migration rates for group selection to promote the evolution of cooperation. If migration/mixing never occurred, group A would outperform group B, but the level of cooperation within group A would continue to decrease because of the within group selection forces favoring defection. In order for the group level selection forces to be strong enough to offset this, there must be a certain amount of mixing, migration or budding of new groups. On the other extreme, consider a situation in which there is a great deal of random migration or mixing. This makes the operation of group-level selection nearly impossible. With large amounts of random migration, all individuals are essentially part of one group, and individual level selection will prevail in such circumstances. In order for group selection to be strong enough to favor the evolution of cooperation, the migration/mixing rate must neither be too large nor too small. As discussed earlier, this mixing or migration is typically parameterized in models of group selection (i.e., it does not emerge from the behavior of agents), and the conclusion reached by these models is that there is a very narrow band of migration rates that can favor the evolution of cooperation via group selection, and that it is therefore unlikely that group selection promoted the evolution of cooperation in human evolutionary history (Richerson & Boyd 1998) and the evolutionary history of other species (Boorman & Levit 1980). These and other conclusions from traditional models are discussed below.
117
4.3. RESPONSIVE MOVEMENT AND GENETIC GROUP SELECTION 4.3.1. Responsive Movement and Individual Level Selection The central focus of this thesis is on the role of responsive movement in promoting the evolution of cooperation. In order to appreciate the simplicity of this process, one must consider that evolution selects for responsive behavior in all organisms, whether animals, plants, or single celled amoebas. Any movement in space influenced by the physical or social environment is responsive movement. For example, whenever an organism moves in response to one stimulus (e.g., toxin) and stays in response to another stimulus (e.g., food), the organism is using responsive movement. Evolution can select for such behaviors based simply on the benefit they provided to the individual. Individual level selection can favor responsive movement and this can actually provide a pathway by which cooperation can evolve via group level selection.
4.3.2. Responsive Movement, Assortment and Levels of Selection As discussed above, responsive movement can easily be selected at the individual level because there are a large number of ways in which it can promote the interests of the individual. Now consider how such a strategy would play out in species in which the cooperative behavior of each individual can affect the payoffs of others. For example, an organism as simple as a bacterium can produce a protein that provides a benefit to any other nearby bacteria, or a group of humans can be engaged in a public goods game in which the cooperativeness of each individual influences the outcomes for every other
118 player. If individuals simply apply the rule of leaving, or ‘Walking Away,’ when they are not receiving adequate benefits from the environment, they will tend to stay in groups that are made up of individuals that provide benefits. If all individuals use this Walk Away rule, the aggregate effect will be that the least cooperative groups will be least stable (as individuals constantly leave them) and more cooperative groups will be more stable (because individuals are more likely to stay in them). If there is any variation in the cooperativeness of individuals, a consequence of this strategy can be an increase in positive assortment, because groups that happened to have more cooperative individuals will be more stable (although defectors using such a rule can actually increase negative assortment under certain circumstances; see Pepper 2007). The assortment that emerges from the behavior of these individuals can then be acted on by group selection in order to favor groups made up of more cooperative individuals. This will cause those groups to increase in size more quickly than less cooperative groups, and although the level of cooperation in a group necessarily decreases over time because of individual level selection acting within the group, when cooperation decreases enough, individuals will begin to leave the group (and new groups may form made up of individuals who left the original group). This type of migration can increase assortment even further, and it can counteract the individual level selective force within groups that leads to constant decreases in the level of cooperation within that particular group. A central claim of this chapter is that this type of migration is qualitatively different from the kind of migration assumed in analytical models. Standard analytical models assume random migration or mixing, which decreases assortment. However, when individuals use the Walk Away rule (leaving groups when overall level of
119 investment in that group is too low) this causes systematic, rather than random, migration. This can then lead to an increase in assortment because more cooperative groups are more stable, and new groups can form from individuals who opt to leave less cooperative groups.
4.4. A MODEL OF GROUP SELECTION AND ASSORTMENT 4.4.1. Agent-Based Models of Genetic Group Selection One of the limitations of standard analytical models of the evolution of cooperation via genetic group selection is the emphasis on aggregate features rather than individual decisions and strategies. These models often adopt unrealistic assumptions for the sake of creating simple and general models, but sometimes fail to consider the impact of these assumptions on the validity of conclusions drawn from the model. For example, the assumption of random assortment as a result of migration and mixing is probably unjustified for a large number of species. Not only is responsive movement in space possible for most organisms, it is adaptive, allowing individuals to escape cost and approach benefits. Further, when individuals adopt a contingent movement strategy, this can easily be applied to the decision of if/when to migrate out of a group or simply when to switch partners (Aktipis 2004; Hammerstein 2003). There are a number of well-known mathematical and simulation models evolutionary models for the evolution of cooperation in groups, several of which employ varieties of multi-level selection (e.g., Boyd & Richerson 1985; Fehr & Fishchbacher 2003; Gintis 2003; Maynard Smith 1964; Wilson 1977; 1979; 1987; Wright 1931). By
120 necessity, analytical models and simulations abstract out features of the environment to make modeling tractable. In particular, variables such as migration rate and the probability of interacting with others are either fixed or varied parametrically. In contrast, it is possible with an agent-based model to examine the impact of individuals’ decision rules on various parameters, including migration, assortment and group size. Because agent-based models focus on the level of the individual decision-maker, the role of individual level adaptations in influencing contingent movement and subsequent assortment are both easier to model and more intuitive. This is not to say that such models and insights are impossible with standard analytical models, only that such modeling is more transparent and often easier to implement in agent-based models. The ‘Walk Away’ model described below is one example of an agent-based model that suggests the important role contingent movement might have played in the evolution of cooperation in groups. Implementing the Walk Away strategy results in endogenously determined migration rates, variable and dynamic group sizes and other interdependent features, making it a poor candidate for analytical simulations. Spatial agent-based models such as the present one have the potential to illustrate in an intuitive and transparent way the principles of multi-level selection: within a group of individuals who exert fitness effects on one another, selection will favor those who do not deliver benefits to the group at a cost to themselves because these individuals will enjoy higher within-group fitness. However, groups that consist of individuals who do deliver fitness benefits within the group can increase in size more quickly than groups with fewer such individuals. If this between-group selection is stronger than withingroup selection, the proportion of altruists in the overall population can increase while
121 the proportion of altruists in each group declines, an effect known as Simpson’s Paradox (Simpson 1951; Sober & Wilson 1998). The importance of between-group selection in the evolution of cooperation in humans and other species has long been an issue of contention (Dawkins 1976/1989; Williams 1966; Wilson 1983; Wilson & Wilson 2007). The dynamics resulting from agent behavior in this simulation demonstrate that the capacity for responsive movement could have played a role in increasing the strength and effectiveness of between-group selective pressures. 4.4.2. The Model The simulation described below is provided as a demonstration of the potential importance of the Walk Away strategy (Figure 4.1) in shaping population dynamics and subsequent selection for cooperation. Agents using the Walk Away strategy stay only in groups that provide sufficiently high returns. The results suggest that individual-level decision rules for responsive movement can play an important role in the dynamics of selection on groups. Further, the basic qualitative findings reported below suggest important directions for the exploration of the ways in which selection at the individual level can influence selection at the level of the group. Results are provided as a demonstration of some of these fundamental principles rather than a thorough analysis of parameter values. Because this model allows for continuous investment with no upper limit, a systematic analysis of assortment is difficult without arbitrarily breaking up the strategies into different types based on their parameters of cooperativeness and Walk Away threshold. However, results from earlier binary models of investment have demonstrated dramatic changes in assortment over the lifetime of a simulation,
122 presumably because of the non-random migration enabled by the Walk Away strategy (Aktipis manuscript). The continuous model provides a more intuitive demonstration of the process of multilevel selection and the role that responsive movement can play in the dynamics of selection (Figure 4.2). Walk Away R >= T
R= T
Move
Stay Give I
R
Figure 4.1. State transition figures illustrate the simplicity of the Walk Away strategy. Boxes indicate states the agent can occupy; arrows indicate possible state transitions. Agents stay in a group if and only if the return (R) received from the group meets or exceeds the agent’s threshold (T). Agents in the “Stay” state contribute “I” to the group each time period and stay on the patch. In the “Move” state, agents move one step each time period and do not contribute. Agents change to the “Stay” state when they encounter another agent (or agents), staying only if the benefit received from the group exceeds their threshold.
In this simulation, agents have two parameters, their level of investment in the ‘group fund,’ I, representing their level of cooperativeness (which is an integer value initially set to a number between 0-10) and their threshold, T (which is an integer value initially set to a number between 0-20). Agents invest I in the group fund, and that amount is doubled and split between all agents in the group, constituting a tragedy of the commons (Hardin 1968) where individuals benefit most from not contributing, but the group does best if all individuals contribute. This structure is also referred to in the
123 analytical and the experimental literature as a “social dilemma” (Dawes 1980) or a “public goods game” (Ledyard 1995). A group is defined as agents currently occupying the same lattice position, and agents leave a group if the return from the group investment (R) is below the agent’s threshold (T), moving randomly (changing their heading slightly during each time period) when not in a group. This model was created in Starlogo 2.0.2 (MIT Media Laboratory 2003). Additional methodological details can be found in Appendix B. In order to understand how the agents’ movement rules increased selection for cooperative (high investing) agents, consider a typical run. In the first phase (Figure 4.2a), agents move around until they land on a patch with a group whose level of cooperation meets or exceeds their threshold. Some groups are made up of agents with higher levels of investment than others. Within each of these groups, the level of investment decreases over time because lower investors bear smaller costs and thus enjoy greater reproductive success. However, groups of high investors increase in size more quickly than groups of low investors, which results in an overall increase in investment. This therefore represents a clear instance of “Simpson’s Paradox,” in which cooperation increases in the metapopulation despite selection against cooperation within each group (Simpson 1951). Next we consider the second phase (Figure 4.2b). Once the most cooperative group (or groups) has increased in size at the expense of the less cooperative groups (often causing these less cooperative groups to go extinct), the forces of between-group selection are small or zero. This means that within-group selection will favor individuals who are less cooperative, investing less in the group while receiving the same return as
124 the more cooperative individuals. Without the capacity for contingent movement (or some exogenously imposed regrouping scheme) investment would continue to decrease until it reached zero. However, because agents have a threshold specifying the minimum return they are willing to accept in order to remain in the group, a particular group will break apart due to agents departing as the level of investment (and therefore the return to each agent) goes down. This leads to the third phase (Figure 4.2c): when agents’ thresholds are reached, they leave the group, often forming new groups on the periphery of the original group, a process here referred to as budding. Once these new groups are established, the between-group forces begin to act again, favoring more cooperative groups, restarting the cycle with the first phase of competition between groups (Figure 4.2d). If the betweengroup selection component is strong enough, the overall level of investment will be higher at the end of a complete cycle, although it often declines during Phase II. The phenomena described in this section can be explored in the interactive simulation at http://www.psych.upenn.edu/~aktipis/MovementGroups.htm.
125 a)
b)
c)
d)
Figure 4.2. Snapshots of typical group dynamics in simulation. Colors range from dark blue (high investment) to green (moderate investment) to red (zero investment) and circle diameter represents group size (ranging from 2 to 50+). Red arrows show moving agents. a). In Phase I, the most cooperative group (darkest blue) is increasing at the fastest rate, due to between-group selection, causing the average level of investment to increase from 9.09 at the time of this screen shot, to 10.36 as this group continues to increase in size. b). In Phase II, this group has out-competed all other groups, allowing within-group selection to act more strongly, and average investment has decreased to 8.25. c). In Phase III, agents begin to leave the target group because the return is below their threshold (investment is 7.76), causing new groups to form. d). This leads back to Phase I, where the most cooperative group grows the fastest again, with average
126 investment increasing up to 8.80 here and continuing to increase as the forces of between-group selection act.
These same basic dynamics are presented in Figure 4.3, where the process by which cooperation increases in the metapopulation is represented visually. Each group decreases in cooperativeness as time goes on, but cooperative groups tend to grow faster than less cooperative groups. New groups are formed as individuals leave, and some of these are more cooperative than the original group, leading to an overall increase in cooperativeness over time. In this simulation, regrouping events occur without any externally imposed assortment or migration. Importantly, the movement rules employed by these agents result in greater assortment, largely because groups made up of cooperators are simply more stable and long-lasting than groups made up of less cooperative individuals. The spatial agent-based nature of this simulation makes this multilevel selection process observable in a way that analytic models do not.
127 Figure 4.3. This schematic illustrates Simpson’s Paradox: the level of cooperation within each group decreases over time while the average level of cooperation increases. This is driven by the constant creation of new groups through agent movement and regrouping as well as the faster rate of growth for the groups that are most cooperative. Group size is indicated by the thickness of the black lines and the thin gray lines show cases where new groups are created because of contingent movement (as a result of agents’ thresholds being reached).
4.5. DISCUSSION 4.5.1. Migration and Differential Stability of Groups In “Adaptation and Natural Selection” Williams argued against the effectiveness and importance of group selection in shaping the evolution of behavior (Williams 1966) and the gene-centered view was proposed as an alternative to group selection (Dawkins 1976/1989). The controversy over the importance of group selection has continued to this day (Dawkins 2008; Wilson & Wilson 2007). The operation of group selection requires semi-isolated populations and some have argued that the population structure in humans and other animals could not support the operation of multilevel selection. The present work suggests that a population of individuals using a Walk Away rule can challenge some of the assumptions about population structure and evolutionary dynamics that have formed the basis for such arguments. Among those who have challenged the viability of genetic group selection as a robust mechanism are Boorman and Levit (1980) and Richerson and Boyd (1998). They cite a lack of differential extinction and weakened group boundaries as the main factors that undermine genetic group selection as an important factor in social evolution. Past
128 work has shown that the Walk Away strategy can support assortment and promote the evolution of cooperation via multilevel selection (Aktipis manuscript) and the present demonstration shows the operation of systematic migration and multilevel selection in an intuitively transparent way. This section describes a number of arguments against the effectiveness of group selection and notes several challenges to these arguments from the present demonstration and previous work on the Walk Away strategy. Systematic migration and differential group stability emerge from agents using Walk Away, changing the dynamics of selection at the group level. 4.5.1.1. Differential extinction of groups Boorman and Levit (1980) claim that differential extinction of groups is unlikely because there are few species with ‘true’ island structures where groups are separated for lengthy periods of time, and that most well demarked ‘groups’ are relatively temporary, existing seasonally or only at the boundaries of the home range of the species. It may well be true that island-like population structures are rare and differential extinction of groups might therefore be uncommon (but see Aviles 1993; West & Herre 1998 for clear examples of species that do have such structures). However, it is not necessary for group selection that groups be completely isolated (especially when migration increases assortment), nor is it necessary for some groups to go extinct for group selection to occur. There need only be differential growth and reproduction of groups and some migration or mixing.
129 4.5.1.2. Permeability of group boundaries Boorman and Levit (1980) also argue that group selection can “generate its own ‘erasure’ principle as the boundaries of groups are rendered more permeable in the course of successful selection” (p. 360). They describe a situation in which group selection for predator defense leads to decreased predator effectiveness and therefore the potential for greater migration between groups that might have otherwise been more isolated. Just because group selection can result in decreased boundaries between groups, this does not necessarily imply that group selection will not be an effective force. As discussed earlier, some migration or mixing is actually necessary for group selection to occur. Further, an increase in migration between groups can actually increase assortment, which could increase the effectiveness of group selection.
4.5.2. Responsive Movement and Cooperation in Humans It has been suggested that the ability to leave current partners and seek out new ones might have played an important role cooperative behaviors in humans and other animals (Connor 1992; Noe & Hammerstein 1994). In humans, it has been shown that humans use something like a Walk Away strategy when given the opportunity in experimental economics settings (Boone & Macy 1999; Hauk 1999; Orbell & Dawes 1993; Yamagishi & Hayashi 1996). To the extent that humans do use such a strategy in the laboratory, this suggests that something like Walk Away might have played a role in the evolution of cooperation in humans. It is suggested here that such a strategy might have promoted the evolution of cooperative tendencies by promoting assortment and increasing the strength of between-group selection.
130 In previous models of multi-level selection, only an intermediate amount of migration and consequent genetic mixing among groups allows cooperation to be sustained. In the present model, cooperation is favored even when migration rate is relatively high. Because evidence in humans suggests that migration rates, assuming random migration, would have resulted in insufficient levels of assortment for genetic multi-level selection to have operated, it has been suggested that between-group selection would not have been a strong enough force to influence the evolution of cooperation in early human groups (Brown & Armelagos 2001; Richerson & Boyd 1998). However, the Walk Away strategy makes migration systematic, not random, increasing positive assortment rather than decreasing it. If humans used a strategy like Walk Away when deciding whether or not to migrate from one group to the next, the possibility of betweengroup selection having acted on humans during our evolution history becomes more likely. 4.5.2.1. Migration and instability in human groups Richerson and Boyd (1998) have argued that populations of early humans were characterized by a structure with a degree of migration and mixing that would have rendered genetic group selection implausible (and they therefore argue that cultural group selection is a more reasonable candidate for an evolutionary process that selected for more cooperative tendencies). Specifically, they argue that the human mating system during our evolutionary history was characterized by intermarriage between groups, sometimes as the result of wife capture, and that this would render deme boundaries too permeable for genetic group selection to have an effect. They also claim that warfare
131 often resulted in the disintegration of a defeated group, and the members of the defeated group often joined other groups where they had bonds of friendship or kinship. They claim that the resulting permeability of group boundaries as the result of such processes works against genetic group selection. However, in the cases of both intermarriage and the break up of groups due to warfare, assortment (with regard to cooperative and group beneficial traits) can be maintained, or perhaps even increased. Presumably, many cases of intermarriage in early human groups were between groups or families that had ties of friendship (or possibly kinship), which makes it possible that these processes could support positive assortment of individuals with cooperative tendencies. There are a number of potential routes to increasing the effectiveness of group selection as a result of warfare. If cooperative groups are likely to be more stable and successful in warfare than less cooperative groups, this can increase the effectiveness of genetic group selection. Further, it might often be the case that less cooperative groups tend to be more heterogeneous (perhaps having recently experienced an invasion of free riders) than other more cooperative groups. In this case, the simple dissolution of such a group can increase assortment. If the more cooperative members can join other preexisting cooperative groups, this would further increase assortment. Also, if groups break up after a defeat and individuals join groups where they have ties of friendship or kinship, this could also increase assortment, making genetic group selection actually more likely. Consider the following situation: group A is made up of individuals who vary in their cooperative tendencies, but they are defeated by a nearby group (B) who wins, perhaps because they are more cooperative overall and
132 therefore better able to generate the resources and manpower to be successful in battle. The individuals in group A (who were likely less cooperative on average than group B) then disperse to the groups where they have ties of kinship and/or friendship. The individuals in group A who were more cooperative might be more likely to have ties to other groups that are cooperative (by virtue of genetic similarly to kin or associations based on reciprocity) and might be more likely to be accepted by those groups than less cooperative individuals after the break up of group A. When individuals join groups in which they have friends and kin, this can have a positive effect on assortment of cooperators with one another. Factors such as migration and group warfare might have promoted assortment in humans and other primates. Therefore, genetic group selection should not be dismissed on the grounds that there was ‘too much’ migration between groups and ‘too little’ stability of groups, because systematic migration and differential group stability could very well have increased the force of genetic group selection by increasing assortment. To summarize, arguments against the viability of genetic group selection as a force promoting the evolution of cooperation tend to rely on arguments about population structure in which it is assumed that migration or mixing will decrease assortment, therefore rendering genetic group selection ineffective. However, assortment can increase as a result of systematic migration. When migration and mixing are not random, the differential stability of groups can play an important role in group selection, with more cooperative groups tending to be more stable and therefore better able to benefit from opportunities for mutual gain.
133 4.5.3. Interactions between Levels of Selection in the Evolution of Cooperation One interesting conclusion suggested by these models is that different levels of selection can interact in order to favor the evolution of cooperation. Responsive movement away from uncooperative groups as described in the Walk Away model can be selected at the individual level because it allows individuals to avoid potentially exploitative groups and often find more cooperative groups to join. These adaptations might even be built on other more basic adaptations designed for avoiding costs and acquiring benefits in the physical world (i.e., foraging adaptations), which have clear individual-level benefits. Individual level Walk Away behavior can give rise to assortment (as a result of more cooperative groups being more stable), which increases the effectiveness of group selection on cooperativeness. In this way, individual level selection can promote group level selection. Agent-based models might provide a distinct advantage when modeling situations in which the decision rules of the agents play such an important role in influencing the effectiveness of group level selection because the cognitive infrastructure for individual level adaptations can be implemented in a straightforward and intuitive manner.
4.5.4. If Genetic Group Selection is Plausible, Why is Cooperation Rare? The present model demonstrates that a strategy as simple as Walk Away can generate population dynamics that promote genetic group selection, suggesting that genetic group selection might be a more plausible mechanism for promoting the evolution of cooperation than has previously been thought. This raises the question of
134 why cooperation is not more common in the natural world. If the conclusions suggested by this model are correct, there are two main possibilities. First, cooperation may actually be common, but we have not yet developed effective ways to systematically measure and understand the vast majority of cooperative interactions in the natural world. For instance, cooperation among microorganisms is a growing area of investigation, and one that has benefited from improved techniques for analyzing small-scale interactions and the genetics underlying cooperative phenotypes (Strassmann & Queller 2007). Second, cooperation might be relatively rare because of physical or social constraints that limit the effectiveness of a Walk Away–type strategy in promoting the operation of genetic group selection. Recall that Walk Away promotes the preferential assortment of cooperators, which promotes group selection. If groups of organisms live in physical or social environments that produce excessive mixing or excessive stagnation, this could limit the effectiveness of the Walk Away strategy in promoting assortment. Also, adaptations on the part of defectors to more effectively follow cooperators or constrain them from moving could limit the effectiveness of the Walk Away strategy, therefore making it less likely that cooperation would be selected via genetic group selection. To the extent that physical and social constraints limit Walk Away's ability to promote the preferential assortment of cooperators, genetic group selection will be less effective at promoting cooperation.
135
4.6. CONCLUSIONS This chapter addresses the question of whether cooperation can evolve via genetic group selection, concluding that genetic group selection might be a more effective force than has been previously appreciated. The potential of individuals to use responsive movement, as in the Walk Away strategy, allows for greater assortment than has been considered with traditional group selection models. This should prompt a reconsideration of the assumption that migration and mixing decrease assortment, because responsive movement promotes systematic migration and differential group stability, which can promote assortment. If migration serves to increase assortment in many cases, this makes the evolution of cooperation via genetic group selection appear much more plausible.
136
CHAPTER 5
CONTINGENT MOVEMENT AND BENEFIT TRANSMISSION IN THE EVOLUTION OF COGNITION AND SOCIAL BEHAVIOR
5.0. ABSTRACT The ability to behave contingently provides important advantages to organisms and genetic lineages that encounter environments with changing features. In this chapter it is suggested that the ability to modify various components of these decision rules based on input from the environment and the ability to generate representations of embedded ecological and social contingent rules can provide additional advantages. However, organisms with contingent rules also become vulnerable to manipulation by entities that emit signals that meet the input conditions of contingent rules. This chapter proposes that contingent decision rules are the foundation for various aspects of cognition and communication. The framework proposed here provides formal tools that might help bridge the gap between genetic and cultural approaches to understanding the evolution of cooperation and other behaviors.
137
5.1. INTRODUCTION The Walk Away strategy is characterized by two main features, the transmission of benefits to entities in spatial proximity and responsive movement away from entities that emit insufficient benefits. The simulations reported in earlier chapters demonstrate that the Walk Away strategy is evolutionarily successful in dyads and in groups under a variety of parameter values. Further, these simulations show that the aggregate effects of individuals using this decision rule in a spatial environment included population dynamics that promoted the evolution of cooperation via group selection. The framework outlined in the present chapter suggests that understanding the evolution of altruism and exploitation might require a careful consideration of the role of contingent behavior. In addition to the contingent movement (Walk Away) rule discussed in earlier chapters, contingent benefit emission is proposed as second essential component of for complex behaviors and interactions. Here it is suggested that basic Walk Away and benefit emission rules form the foundation for more complex adaptations for social behavior, such as those underlying learning, communication, competition, manipulation and parental investment. This chapter joins together diverse phenomena in a conceptual framework that can guide future theoretical work, future simulations and generate empirical predictions. In the interests of laying groundwork for such future work, the decision rules underlying contingent behavior are computationally specified. The input conditions for contingent rules, the structure of the decision rules, and the behavioral outputs are described, providing a formal language in which to understand and explore them. Several areas for
138 future exploration are noted throughout, such as connections to signal detection theory (Green & Swets 1966), optimal foraging (MacArthur & Pianka 1966), parental investment theory (Trivers 1972) and parent-offspring conflict (Haig 2002). The decision rules described here could be implemented in agent-based simulations such as the SIMPLE (Simulated agent Interaction through Movement and Production in a Local Environment) model presented in the first chapter or in variations of the Walk Away simulations presented in Chapters 2-4. This framework can be used to generate predictions about the kinds of contingent behaviors that are likely to evolve in various physical and social environments. However, the primary goal of this chapter is not outline predictions but rather to demonstrate that an approach focusing on contingent behavior and signaling demonstrates connections among a diverse set of behaviors that have not been previously considered as manifestations of the same general principles.
5.2. CONTINGENT BEHAVIOR Contingent behavior is based on the ability to take in information from the environment, operate on that information and differentially respond to it. The present section focuses on simple non-strategic behavior, in particular contingent rules for movement and benefit transmission that make use of information in the environment. As noted in Chapter 1, the capacity for contingent benefit transmission (i.e., contingent cooperation, contingent altruism) has been addressed extensively in the work on reciprocity (Axelrod 1984; Axelrod & Hamilton 1981; Bowles & Gintis 2004; Boyd & Richerson 1988; Fehr et al 2002; Gintis 2000; Nowak & Sigmund 1998a; b;
139 Panchanathan & Boyd 2003; Panchanathan & Boyd 2004; Trivers 1971), memory for past interactions (Aktipis 2006; Cox et al 1999; deVos & Zeggelink 1994; Mealy 1996; Milinski & Wedekind 1998), reputation (Milinski et al 2002), gossip (Nakamaru & Kawata 2002), and indirect reciprocity (Nowak & Sigmund 1998a; b; Panchanathan & Boyd 2003; Panchanathan & Boyd 2004). Contingent movement, however, has received comparatively little attention. This is largely due to the fact that many influential models of the evolution of cooperation have no spatial component (e.g., Axelrod 1984; Axelrod & Hamilton 1981; Gubernick 1981; Hamilton 1964a; b; Trivers 1971) and those that do often to model spatial relations as relatively passive, changing only through birth and death of agents occupying cells (e.g., Killingback & Doebeli 1996; Nowak & May 1992). However, cooperation between entities in the natural world can often manifest in spatial behaviors such as foraging, mating, reproduction, parental care, migration, grooming, predator/prey dynamics and territoriality (Chisholm 1996; Gubernick 1981; Mendoza et al 2002), to name a few. To the extent that cooperation between entities manifests in spatial behaviors, mechanisms for minimizing exploitation and promoting mutual gain might be expected to take advantage of the spatial components of the interactions by, for example, maintaining selective proximity to certain individuals. Recent work on the evolution of contingent movement has shown that contingent movement rules can promote the success of agents who leave regions or social partners that provide insufficient benefits. This can be the result of environmental feedback from an exploited foraging region (Pepper & Smuts 1999; Pepper & Smuts 2002) or based on past behaviors of partners in dyadic (Aktipis 2004; Hamilton & Taborsky 2005) or group-wise (Aktipis manuscript)
140 interactions. Others have investigated the ability to exit from interactions without modeling this as contingent movement in a spatial world (Ashlock et al 1996; Connor 1992; Cox et al 1999; Enquist & Leimar 1993; Eshel & Cavalli-Sforza 1982; Hamilton & Taborsky 2005; Noe & Hammerstein 1994; Vanberg & Congleton 1992). The present chapter is a natural extension of the ideas about assortment, information processing and contingent behavior outlined in the first chapter. The first chapter proposes a novel framework for thinking about models of the evolution of cooperation that is based around the dual themes of spatial assortment and temporal assortment. Because organisms can benefit from the ability to contingently respond to information about temporal and spatial assortment, the ability to processes and use this information in guiding behavior evolved, promoting active spatial and temporal assortment. More specifically, contingent movement is proposed to have evolved because it can lead to more effective exploitation of the environment when resource distributions changed over time (changes in the payoffs of temporal assortment), and contingent benefit transmission is proposed to have evolved because it improved inclusive fitness when the proximity of kin changed over time (changes in spatial assortment). When a trait positively covaries with fitness, it will increase in frequency in the population (Price 1970). The ability to use contingent behaviors such as contingent movement and contingent benefit transmission can covary with fitness because they enable an organism to change its behavioral output when environmental inputs change. This can make it possible for the organism to realize greater benefits and fewer costs than would be possible with a fixed strategy. The language of fitness covariance is used
141 throughout this chapter because it does not entail past selection on a trait in the way that the phrases ‘adaptive design’ and ‘selective history’ do, and it allows the fitness effects of a trait or rule to be (theoretically) decomposed into positive or negative components from different environments or inputs. In fact, the use of a rule in even a single instance can be discussed in relation to the covariance of that rule with fitness in that particular environment or with a particular set of inputs. When organisms are quickly coevolving in response to others (Van Valen 1977), as is proposed here, the ‘forward-looking’ language of fitness covariance and the conceptual clarity that it affords in decomposing fitness effect can make it preferable to alternatives. It is worth making a few additional notes about language and notation in this chapter. Information input/output is represented by the letter I, and the thresholds of contingent rules are represented by the letter T. Throughout this chapter, various subscripts are placed on the information (I) and the threshold (T). The subscripts SA and TA of (I) denote spatial assortment information and temporal assortment information, respectively.
5.2.1. Contingent Movement Contingent movement rules can operate on information about the likely future returns from staying or leaving a location. When an individual remains in the same spatial location for multiple time periods, this is conceptualized as temporal assortment, or proximity in time of an entity with itself or copies of itself (see Chapter 1). A direct source of information about future returns from staying or leaving a particular area might be the present rate of return that an organism is achieving by consuming resources in that
142 area. It could also be the case that individuals developed sensory capabilities that enabled them to ascertain the energy available in the present location and alternative locations (as in the benefit approach rule described in the first chapter). The Walk Away rule makes use of only the present rate of return; it is the simplest contingent movement rule. 5.2.1.1. Walk Away The ability to move away from costs and stay in regions with benefits is a behavior with tremendous adaptive value. Staying close to benefit producing entities can allow for the accumulation of energy and avoiding cost producing entities allow individuals to conserve energy. This energy can then be converted into increased survival or reproductive effort, resulting in more (or more successful) offspring. Arguably, this Walk Away ability might have been the first responsive behavior that evolved in a variety of organisms. The ability to engage in responsive movement does not require complex cognitive abilities, only the ability to respond contingently to the environment (which could be executed by a simple nervous system, regulatory portions of the genome, or even simple receptor dynamics). The Walk Away rule can formally be represented by the general conditional rule below. If (ITA < TTA), Then Emit M ITA = temporal assortment information input from the environment (here the current energy available) TTA = the temporal assortment information threshold of the agent M = the amount (or type) of movement
143 In this decision rule, ITA is the information being processed from the external environment, the inequality ITA < TTA is the decision rule and M is the output of the system. Figure 5.1a also illustrates this rule, denoting the positive fitness covariance of the Walk Away rule with the color blue. 5.2.1.2. Contingent Benefit/Cost Emission Contingent rules for emitting benefits or costs take as input information about spatial assortment (i.e., the spatial proximity of gene copies) (Chapter 1). This information can take the form of any cue correlated with shared genes. This section provides formal descriptions of benefit and cost emission rules including the input, decision rules and output. 5.2.1.2.1. Benefit Emission The transmission of benefit to individuals who are likely to share genes (coding for the cooperative trait) is the essence of theories of kin-based altruism (Hamilton 1964a, p. 118; 1964b). When spatial proximity to kin results from local reproduction and low dispersal, this can potentially favor the evolution of non-contingent benefit transmission. However, non-contingent benefit transmission can be exploited by unrelated recipients who are able to position themselves to take advantage of the benefit emission. Contingent rules for emitting benefits can provide mechanisms for limiting the exploitation of emitted benefits by non-related individuals. Any information that is statistically associated with the presence of those who probabilistically share genes (e.g., temporal cues, spatial cues, pheremonal cues, olfactory cues, visual cues, tactile cues, vestibular stimulation, etc) could be used in a benefit
144 emission rule to decrease the likelihood of exploitation. Tag-based altruism models (where individuals cooperate only with those having a particular trait or tag) capture this basic idea, but the rule described below is more general, allowing for any kind of signal (including, for example, signals from the environment rather than the interaction partner). A formal representation of a basic rule underlying the emission of benefits is described below (see also Figure 5.1b). If (ISA > TSA), Then Emit B ISA = the spatial assortment information, i.e., the likelihood of shared genes TSA = the threshold for spatial assortment information B = the level of benefits emitted In this decision rule, ISA is the information being processed, the inequality ISA < TSA is the decision rule and B is the output of the system.
145
Figure 5.1. a) Walk Away monitors the rate or return, leaving regions in which this input (ITA) has fallen below threshold. This can enable the organism to avoid costs associated with extended interactions with a degraded environment, leading to positive covariance of the rule with fitness. b) The benefit emission rule monitors cues associated with the likely spatial proximity of shared genes (I SA), emitting benefit when this value is above threshold. This contingent rule can result in the organism providing benefits to entities that share genes (and therefore probabilistically shares coding for the contingent rule), leading to positive covariance of the rule with fitness.
5.2.1.2.2. Cost Emission Just as organisms might be expected to contingently emit benefits when in proximity to likely kin, they would also be expected to contingently reduce the emission of costs such as the creation of harmful byproducts and exploitation of the environment.
146 For example, feeding restraint can be conceptualized as an altruistic behavior because it leaves benefits in the environment to be consumed by others, and this trait is more likely to evolve among kin who share a local environment. More controversial is the idea that individuals could evolve to be spiteful, engaging in costly actions in order to harm other organisms. However, simulations have shown that spiteful behavior can be favored under certain circumstance because it can enable invasion of other strategies in space (Nakamaru 2006). Rules underlying the ‘spiteful’ emission of costs could similarly be tethered to signals of shared genes, for example, enabling an invader to impose costs on unrelated competitors, clearing a way for their own offspring, on whom they would not impose costs. For example: If (ISA < TSA), Then Emit C ISA = the spatial assortment information, i.e., the likelihood of shared genes TSA = the threshold for spatial assortment information C = the level of costs emitted
5.3. THE CALIBRATION OF THRESHOLDS FOR CONTINGENT BEHAVIOR The previous section describes the simplest manifestations of contingent behavior, contingent movement and benefit transmission rules that take into account input from the environment in a single time period, comparing that information to a fixed threshold. However, the thresholds used by agents can potentially be altered based on information
147 input from previous time periods. In this section the most simple calibration rule is discussed, one in which the threshold is calibrated based solely on the information input from the most recent time period. Walk Away and benefit emission rules are described in which information that was input in the most recent time period is compared to input from the present time period.
5.3.1. Calibration of Walk Away based on previous Temporal Assortment Information Recall that temporal assortment information is information about the beneficial or harmful effects of an organism’s extended coupling with the environment (leading the organism to affect its future self through the shared environment, see Chapter 1). All organisms occupy space and must consume resources from their local environments in order to survive, grow and reproduce. If the rate of replenishment of the resource is slower than the pace of consumption, an organism staying in the same spatial location faces a future of decreasing resource availability and might benefit from movement in space. Below a simple variation of the Walk Away rule is described that allows an organism to selectively leave regions in which the rate of return from their local environment decreases between the previous time period and the current one The basic Walk Away rule operates on temporal assortment information, as does this variation of the rule. However, the present rule takes two pieces of information about temporal assortment (from the past and present time periods) and compares them in the decision rule, while the original Walk Away rule takes only one piece of information about temporal assortment and compares it to a fixed threshold value. This is formalized
148 below, with the threshold (TTA) for the next iteration of the rule being set to the value of the most recent input (ITA). This allows the organism to compare the present input (ITA) with the past input (which is now TTA). This makes the decision rule: If (ITA < TTA), Then Emit M Set TTA = ITA ITA = temporal assortment information input from the environment (the current energy available) TTA = the temporal assortment information threshold of the agent M = the amount (or type) of movement In this decision rule, ITA is the information being processed from the external environment, the inequality ITA < TTA is the decision rule and M is the output of the system. The line ‘Set TTA = ITA’ sets the threshold for the next iteration to the input from the present time period, allowing for a comparison between time periods. Figure 5.2a demonstrates this process.
5.3.2. Calibration of Benefit Emission based on Spatial Assortment Information The basic benefit emission rule operates on spatial assortment information, emitting benefits only when this information exceeds a fixed threshold. This rule can be modified to allow the threshold to be calibrated based on input from the environment, a possibility explored below.
149 The rule described below is analogous to the above Walk Away rule in that it simply sets the current threshold to the input from the previous time period, allowing for a comparison between the present input and the most recent input. For the benefit emission rule, this simply means that an organism would emit benefits when the present signals of kin proximity are higher than they were in the most recent time period. In other words, the emission of benefits is conditional on a positive change in proximity of kin. If (ISA > TSA), Then Emit B Set TSA = ISA ISA = temporal assortment information input from the environment (the current energy available) TSA = the temporal assortment information threshold of the agent B = the level of benefits emitted
In this decision rule, ISA is the information being processed from the external environment, the inequality ISA > TSA is the decision rule and B is the output of the system. The line ‘Set TSA = ISA’ sets the threshold for the next iteration to the input from the present time period, allowing for a comparison between time periods (see Figure 5.2b).
150
Figure 5.2. a) The Walk Away threshold (TTA) can be set to the input from the previous time period (ITA) from t=0, making the rule sensitive to a decrease in ITA between t=0 and t=1, in other words, whether the organism is experiencing decreasing returns. This can enable the organism to avoid costs associated with extended interactions with an environment that is decreasing in quality, leading to positive covariance of the rule with fitness. b) Similarly, the benefit emission threshold (TSA) can be set to the input from the previous time period (ISA) from t=0, making the rule sensitive to a decrease in I SA between t=0 and t=1, which would make benefit emission contingent on information about an increase in the spatial proximity of kin. This can lead to positive covariance of the rule with fitness.
Information from the past is important to the extent that it can predict the likely future outcomes associated with alternative behaviors. Here it is suggested that the ability to effectively identify and contingently respond to information that predicted
151 future resource availability (i.e., changes temporal assortment information) and the payoffs from benefit emission (i.e., changes in spatial assortment information) would be expected to strongly covary with fitness. It has elsewhere been noted that the organisms should evolve to identify and incorporate into decision rules the relevant information in the environment based on evolutionary considerations (Gallistel 2000; 2002). The threshold calibration rules described above can be considered a starting point for investigating the adaptive value of small increase in complexity to the Walk Away and benefit emission rules. The decision rules described here might, at first, seem so simple as to provide little traction for producing adaptive behavior in the face of complex optimization problems such as those underlying foraging (in alternative local environments) and reproductive investment (in alternative mates/offspring). The effectiveness of these simple rules will be dependent on the structure of the environment and the payoff from alternatives. Future simulations will allow for the systematic exploration of the adaptive value of these rules in alternative environments.
5.4. CALIBRATION OF OUTPUT When an organism is likely to encounter new contingencies between their behavior and resultant outcomes in its lifetime, it can be advantageous to calibrate behavioral output based on feedback from previous time periods rather than having fixed input/output rules. Different behavioral output can result in different payoffs for the organism, and these payoffs can be processed by the organism as input, guiding the system to produce beneficial outputs.
152 In this section it is suggested that the calibration of movement output (M) and benefit emission (B) can provide advantages to organisms, leading to selection for the capacity to modify the output based on feedback from the environment. These rules can provide clear advantages if organism can engage in behaviors (M or B) that have causal effects on fitness payoffs for that organism in future time periods which are detectable as feedback/input (I) for that organism. However, these advantages cannot be realized if noise is prohibitively high, the organism otherwise unable to detect information (to use as feedback/input) that correlates with fitness payoffs, or the organism cannot change its payoffs by changing behavioral output. This framework predicts that the ability to calibrate outputs will be positively correlated with systematic characteristics of the organism and its environment that allow the organism to effect its own payoffs and detect information about those contingent payoffs.
5.4.1. Calibration of movement The Walk Away rule described in earlier sections generates movement when the return from being in a particular location goes below the agent’s threshold. In the natural world diverse movement patterns are often possible for organisms, and these various movements can have different payoffs. If organisms are able to continue engaging in movement patterns (M) that are quickly followed by beneficial outcomes such as the acquisition of resources (through the input, ITA, exceeding the threshold, TTA), this can provide higher payoffs than a fixed strategy. A decision rule that allows the output (in this case, M) to be calibrated is described below. The organism emits a movement (M), keeping the value of M the same if the outcome, i.e., the input (ITA) is above the threshold
153 (ITA), and modifying M slightly (Set M = M ± Δ) if the input (ITA) is below threshold. This process is also illustrated in Figure 5.3a. Emit M If (ITA < TTA), Set M = M ± Δ 5.4.2. Calibration of benefit emission Earlier it was suggested that benefit emission rules operate on information from the environment about the likelihood that copies of genes will benefit from benefit emission (which can be a result of simple spatial proximity to kin or the result of directed transmission). Organisms can benefit from using a rule to continue to emit benefits when receiving feedback (input, I) that correlates with the positive fitness effects of these benefits on kin. If the input goes below a threshold, the organism can change the benefit emission output. This rule described below parallels the movement calibration rule above (see also Figure 5.3b): Emit B If (ISA < TSA) Set B = B ± Δ The essence of this rule is simply that an organism could vary benefit emission to a particular entity, settling on the level of benefit emission that provides feedback (input) indicating sufficiently positive marginal benefits to kin. For example, an organism might vary benefit emission to an offspring, settling on a value that provides input correlated with high marginal returns from that level of parental investment. This input function for the decision rule would take multiple inputs including cues about shared genes with the
154 receiver of the benefits, the marginal benefit to the receiver from the transmission, and the marginal cost to the sender for emitting the benefit. If combined with an appropriate output generation rule (e.g., a threshold calibration rule) this rule could potentially allow for the calibration of the level of benefit emission that realizes high marginal increases to reproductive success to from investment in kin.
Figure 5.3. a) This figure illustrates how the movement output of Walk Away rule (M) can be varied (Set M = M ± Δ) if the return (ITA) is below the threshold (TTA). At t=0, the organism emits M, resulting in low ITA which does not meet the threshold. If the return is above threshold, the organism continues to emit the same output (M). b) In parallel, the output of the benefit emission rule can be varied (Set B = B ± Δ) if the
155 return (ISA) is below the threshold (TSA). If the return is above threshold, the organism continues to emit the same output (B).
The framework proposed above demonstrates that a variety of phenomena that have been considered disparate can be considered manifestations of general principles of contingent behavior and calibration of contingent systems. For instance, these kinds of decision rules can potentially lead to phenomena related to learning and conditioning (e.g., Gallistel 2003; Gallistel & Gibbon 2000; Rescorla & Wagner 1972). The decision making systems underlying optimal foraging (MacArthur & Pianka 1966), and optimal parental investment (Trivers 1972) can also be considered in this framework. These are topics that will be further explored in future work.
5.5. THE VULNERABILITY OF INPUT CONDITIONS In earlier sections it is suggested that both contingent movement and contingent benefit transmission systems are designed to operate on various information from the environment. Contingent movement systems can operate on temporal information in order to increase the likelihood of acquiring resources from the environment (e.g., Walk Away), while benefit emission rules are designed to process information correlated with spatial assortment (of genes), increasing the likelihood that benefits will be transmitted to other entities in a way the promotes the success of that entity’s genes. The present section describes how the input conditions for contingent movement and benefit emission rules can be exploited by other organisms, leading to the evolution of cooperative and manipulative signals and complex processing systems on the part of receivers.
156 The work on signaling and communication has yielded no real consensus on the proper definitions for terms such as ‘signal’ and ‘cue’ (Hasson 1994; Hauser 1996; Krebs & Dawkins 1984; Maynard Smith & Harper 2004). For the present purposes, the term ‘passive signal’ is used to refer to ecological or social information that meets the input conditions of a receiver without there having been any selection on the entity for emitting the signal, and the term ‘active signal’ is used to refer to signals which have features that evolved because they generated positive fitness covariance for the sender. There are some similarities between definitions of passive/active signals and the definitions of signals and cues that have been offered by certain authors. The definition of passive signals noted above corresponds to the definition of cues as “feature[s] of the world, animate or inanimate, that can be used by an animal as a guide to future action” (Hasson 1994; Maynard Smith & Harper 2004). The definition of active signals offered above is similar to the description of signals offered by Hasson as well, “Unlike other traits, signals earn their positive effect from their potential to immediately change the behavior of other individuals” (Hasson 1994). In contrast to other definitions of signals (e.g., Maynard Smith & Harper 2004), the present definition of active signals does not require that the transmission of information between parties benefit the receiver as well, only that the signaler benefits from changing the behavior of the receiver and therefore experiences selective pressure for more effective signaling. The definitions of active and passive signals offered here therefore include the possibility of negative effects on the receiver, as in the transmission of manipulative/deceptive information. The question of whether or not ‘signals’ or ‘communication’ must be cooperative has been treated extensively by Hasson (1994), who notes that excluding cheating or deceptive signaling
157 in definitions of signaling or communication contributes to a loss of generality about information transmission. In the interests of providing definitions of terms that capture the generality of information transfer from one entity (who benefits) to another (who might or might not benefit) and promote conceptual clarity by distinguishing between passive and active information transmission, the terms ‘passive signal’ and ‘active signal’ are introduced and used throughout this chapter. Evolution favors traits that co-vary with fitness (Price 1970), whether these traits are fixed or adaptively contingent on environmental input. The basic thesis of the first chapter is that organisms evolved to use information in the environment as input to contingent systems for movement and benefit transmission because the ability to do so positively covaried with fitness. Once these decision making systems were in place, the input conditions could then be used by other organisms, leading to changes in the relationship between the contingent rule and fitness (therefore influencing subsequent selection). The presence of higher fidelity ‘honest’ signals can positively affect the covariance between the contingent rule and fitness while the presence of more effective mimics that emit ‘deceptive’ signals negatively affects the covariance between the contingent rule and fitness. In other words, there can be both antagonistic ‘red queen’ coevolution (Van Valen 1977) in signaling systems where there are conflicting interests and cooperative coevolution when signalers and receivers share interests. It is suggested that the capacity for active signaling evolves when an initially passive signal probabilistically meets the input conditions of the receiver’s contingent rules for movement or benefit emission, resulting in an outcome that benefits the sender. Evolution selects for features that increase the likelihood that a passive signal will get
158 processed by the receiver, resulting in an active signal that meets the input conditions for receiver’s contingent movement or benefit emission rules. The processing of this signal can have systematic effects (either positive or negative) for the receiver’s fitness, leading receivers to evolve to become more sensitive to the signals (when there is positive covariance of the contingent rule and fitness) or less sensitive to the signals (when there is negative covariance of the contingent rule and fitness). It is suggested that signaling originates when initially passive signals happen to meet the input conditions of receiver’s contingent rules (which have originally evolved for processing information about temporal and spatial assortment). Active signals evolve when these passive signals happen to meet the receiver’s input conditions for contingent movement or benefit emission in a way that generates positive effects for the sender. These active signals might be cooperative, amplifying the information content of previously passive signals (decreasing uncertainty for the receiver) or manipulative, degrading the content of a previously passive signal (increasing uncertainty for the receiver). This is in contrast to other approaches which specify that signals must benefit receivers (Krebs & Dawkins 1984; Maynard Smith & Harper 2004). New notation is introduced in this section. When the information being transmitted from one entity to another is not bracketed, this represents a passive signal. Organism process unbracketed information as if it is unconditionally true (for a similar treatment, see Cosmides & Tooby 2000a). When information input or output is enclosed within brackets preceded by the letter S (e.g., S [ITA]), this indicates an active signal about certain information. In active signaling the information has been operated on by some (evolutionary or cognitive) process that changed the statistical or causal
159 relationship between input/output and that state of the world or the organism emitting the signal.
5.5.1. From passive to active signals Passive signals are essentially just information from the environment (i.e., spatial or temporal assortment information) that meets the input conditions of a receiver’s processing systems. Consider, for example, an organism that develops the ability to contingently approach sites or prey that emit passive signals (e.g., resource gradients, coloration, movement) that indicate their potential resource value. This can be generalized as a benefit approach rule which uses passive signals correlated with likely resource value rather than using information from (for example) consumption history to judge resource value. If (ITA > TTA), Then Emit M ITA = passive signal from the environment containing temporal assortment information (likely resources) TTA = the threshold for this passive signal of temporal assortment M = the amount or type of movement If the effect of an organism’s behavior is neutral with regard to the fitness of the entity being consumed (or the resource being consumed is not an evolving entity such as light or water), the information processed by receivers will continue to be a passive signal. This is shown in Figure 5.4a (the neutral effect on the resource entity is denoted
160 with the color green). However, if the organism’s behavior has a positive or negative effect on the entity being consumed, then there will be selection to make the signal more conspicuous or more cryptic, respectively. For example, fruits that rely on animals for seed dispersal have evolved to be more conspicuous and attractive due to positive fitness covariation for being detectable (Figure 5.4b), while prey who experience massive fitness decrements from detection and consumption evolve cryptic coloration and features that to make them less likely to be detected by predators due to negative fitness covariation (denoted by the color red, Figure 5.4c) for being detectable. The emission of active signals regarding temporal information is denoted with S [ITA], indicating that it is a signal about temporal assortment information. If an organism lacks the ability to detect the fact that the transmission is an active signal (e.g., S [ITA]), it will simply be processed as a passive signal (e.g., ITA), and enter into the contingent rules that take such information as input. This is assumed to be the case unless otherwise noted. The ability to properly identify and operate on ‘bracketed’ information likely involves the operation of a number contingent rules, but the design of this ‘debracketing’ system is not a topic addressed in this chapter.
161
Figure 5.4. a) A signal is considered passive if it meets the input conditions of a receiver’s contingent rules, yet its emission has neutral or null effect on the sender’s fitness. b) If the sending of a signal has a positive effect on the sender’s fitness, it is considered a positive signal, and selection will act to make this signal more conspicuous and clear so that it meets the threshold of the receiver (ITA> TTA). c) In contrast, a negative signal is one that has a negative effect on the sender’s fitness. Selection will act to make this signal less conspicuous so that it will be less likely to meet the threshold of the receiver (ITA> TTA).
162 When the transmission of the signal positively covaries with fitness and the receiver’s ability to process the signal also positively covaries with fitness, the signal will evolve to be more conspicuous and the receiver’s processing systems will evolve to effectively detect the signal. In general, when the interests of two entities are completely aligned there will be no conflict in the transmission of a signal from a sender to a receiver (Maynard Smith & Harper 2004). This no-conflict situation obtains whenever the sender cannot obtain greater fitness covariance as a result of the signal transmission than the receiver (for example, asexual clones or identical cells in a multicellular organism). The relationship between a signaler and receiver with shared interests is represented above (Figure 5.4b). However, interaction partners often have divergent interests. If an entity sending a passive signal is harmed by the sending of that signal (c), the entity will evolve to reduce that passive signal (e.g., limiting conspicuous movement) or emit a signal that makes it less conspicuous (e.g., crypsis). Situations in which organisms have some conflicting interests, but can still benefit from information transfer, can, under certain circumstances, enable the evolution of honest signals with the signaling system converging on cues that reliably correlated with features of the sender (i.e., an index or a costly signal). These situations (with both conflicting and cooperative interests) have been addressed in other work (e.g., Bergstrom & Lachmann 1997; Johnstone & Grafen 1992; Lachmann & Bergstrom 1998; Maynard Smith 1991; 1995; Maynard Smith & Harper 2004; Zahavi 1977), but might benefit from the a treatment according the framework presented above.
163 5.5.2. Honest and deceptive signals Another situation in which there are divergent interests is that which occurs when one entity exploits the contingent rules of another; in other words, when a sender transmits a signal that manipulates the behavior of the receiver in a way that benefits the sender (at the detriment to the receiver). To the extent that the receiver incurs costs from being manipulated by signals that meet the input conditions of their contingent rules, this will negatively affect the covariance between the contingent rule and fitness. Receivers can encounter both honest and deceptive signalers. The evolution of the discernment system of receivers will be influenced by a number of factors, which can be understood in the context of signal detection theory (Green & Swets 1966). The ratio of encounters with each type, the ability to distinguish them (i.e., the effectiveness of the input conditions), the costs of both types of errors and the benefits of correct discernment all affect the overall covariance of fitness with the receiver’s contingent rule. Discernment systems that provide the highest covariance with fitness will be selected. The importance of signal detection theory for understanding the evolution of animal signaling has been noted elsewhere (Hauser 1996; Wiley 1994), and can be explored in greater depth in future work. These ideas are also captured in the notion of Batesian mimicry (Bates 1863). This approach considers three entities, the model (who emits ‘honest’ signals), the mimic (who ‘deceptively’ copies the signals emitted by the model) and the dupe (who ‘mistakes’ the mimic for the model) (Pasteur 1982). The resulting evolutionary arms race favors the receiver (dupe) developing more keen discernment (e.g., through changes in the input conditions or complexity of the decision rule), the model providing more
164 sophisticated signals, and the mimic copying these signals. In other words, mimics evolved to exploit the decision making systems of others, and this increased selection for more sophisticated decision rules and more sophisticated signaling systems. As selection acted to increase the complexity of these systems in order to avoid the costs associated with exploitation by mimics, signal and signal processing systems came to have features that we associate with ‘lock and key’ systems. Over time, signal mimicry leads to selection for decision rules with input conditions that were increasingly sophisticated locks (for discerning honest from deceptive signals), and the signals emitted by models (and mimics) became more complex keys. In the remainder of this section it is suggested that the input condition of these contingent rules can be exploited by organisms that can benefit from manipulating a receiver’s contingent movement or benefit transmission. This approach has broad similarities to other evolutionary approaches to signaling and communication, but differs in suggesting that the structure of these signaling interactions is most effectively conceptualized as senders using the input conditions of receiver’s contingent rules, rather than focusing on whether the sending of information results in mutual benefit (Hauser 1996; Krebs & Dawkins 1984; Maynard Smith & Harper 2004). Whether information emission/transmission results in cooperative or exploitative interactions is likely to vary based on the relationship between interactions partners and the selective history on signal transmission. There are also likely to be many cases of signal transmission in which there are both cooperative and competitive elements, topics treated more extensively later in this chapter.
165 5.5.2.1. Exploiting contingent movement As noted at the beginning of this section, organisms can transmit signals that make use of the contingent rules of other organisms. This can take the form of mutually beneficial signal transmission or signal transmission that benefits the sender at the expense of the receiver. An example of mutually beneficial signal transmission is described in Figure 5.5a. Consider fruits and the animals that consume them and consequently disperse their seeds. The fruit’s signal preserves the relationship for which animals’ perceptual systems were designed (to detect and approach resources). This honest signal is stable because both animals and fruits can benefit from an animal detecting (ITA> TTA), approaching (M), consuming (benefiting the animal) and eventually dispersing the seeds of a fruit (benefiting the fruit). Other signals evolved to manipulate receivers for the sole benefit of the sender. For example, the angler fish attracts prey via a noodley appendage resembling a worm (Maynard Smith & Harper 2004). The predator emits signals that exploit the prey’s contingent movement rules (i.e., the benefit approach rule of the prey). The worm-like appendage emits signals that meet the input conditions of the prey, resulting in movement towards the entity emitting the signal. Figure 5.5b demonstrates how a signaler (in this case a predator) can exploit the contingent movement rules of the receiver by using its lure to meet the input conditions of prey that were designed for processing signals of resource availability.
166
Figure 5.5. a) A signal is considered honest if it preserves the statistical relationship for which the receiver’s input conditions were selected, resulting in positive fitness effects for the receiver for using the contingent rule. For example, fruits can benefit from signaling their resource value to passing animals, increasing the likelihood of seeds eventually being dispersed. b) A signal is considered deceptive if it violates the statistical relationship for which the receiver’s input conditions were selected, resulting in negative fitness effects for the receiver for continuing to use the contingent rule. The angler fish, for example, deceptively signals resource value via the worm-like lure, meeting the input conditions (I TA> TTA) of the receiver, leading the receiver to approach (M), which makes it more likely that the prey will be consumed by the angler (benefiting the sender at the detriment of the receiver).
167 In many other situations organisms can benefit from manipulating the movement of other organisms. For example, prey can benefit from predators not approaching; mates can benefit from partners approaching/staying and offspring from parents approaching/staying, among others. Broadly, if the payoffs for one organism depend on the spatial behavior of another, it can benefit from emitting a signal, S [ITA], that changes the movement output of the other organism. To the extent that the receiver bears costs from having its spatial behavior manipulated, evolution will act to increase the sophistication of the system, thereby decreasing the likelihood that the receiver will accept input from the sender. In contrast, if the receiver benefits from the processing of the signal, evolution will act to increase the likelihood that the input will be processed. 5.5.2.2. Exploiting contingent benefit transmission An organism’s contingent benefit transmission rules can similarly be used for the purposes of exploitation or mutual benefit. For instance, the ability of offspring to signal their identity as close kin, enabling parents to selectively transmit benefits, could benefit the inclusive fitness of parents and offspring. In clonal species the shared interests of parent and offspring can favor the evolution of accurate signaling systems. In sexually reproducing species, parents and offspring may share interests only up to a point, with parent-offspring conflict emerging after a certain level of parental investment (Haig 2002). Nevertheless, parents and offspring share some interests when it comes to the parent identifying its offspring, enabling it selectively transmit benefits to offspring (see Figure 5.6a) at least up to the point where the marginal benefits to the parent’s reproductive success favor investment in the offspring.
168 By definition, organisms can benefit from other organisms transmitting benefits. Organisms can therefore be expected evolve to emit signals that are designed to activate the benefit emission system of receivers. A well-known example of mimicry, and one relevant to contingent benefit transmission, is that of brood parasites such as the domestic cuckoo (e.g., Johnsgard 1997). Cuckoos lay eggs in the nests of other bird species with patterns that match the species typical egg. Hatching cuckoo offspring then grow quickly, evict the other chicks from the nest, and provide visual and auditory cues that encourage the unknowing foster parent to transmit benefits in the form of regurgitated food. The signals emitted by cuckoo eggs and chicks exploit the input conditions of the contingent benefit transmission (i.e., parental investment) decision rules used by parents for interacting with offspring (to transmit benefits to viable offspring). This situation is represented in Figure 5.6b.
169
Figure 5.6. a) If offspring are capable of signaling (S [ISA]) their status as sharing genes with a parent in a way that reliably meets a parent’s input conditions (I SA> TSA), for emitting benefits (emit B), this can directly benefit the offspring and benefit the parent’s inclusive fitness via its effect on the offspring, making an honest signal relatively stable (over investment levels in which there is no parent-offspring conflict). b) An offspring mimic capable of emitting signals that meet a parent’s input conditions (ISA> TSA) for benefit transmission (emit B), can benefit itself at the expense of the foster parent.
Other aspects of contingent rules might be subject to manipulation by senders. For example, rules that compare current input to the last input (i.e., threshold calibration rules) could be exploited by organisms that send a sequence of signals (for example, first a low value and then a high value). Interestingly, a well-know bias in the literature in human decision making known as the ‘anchoring effect,’ describes essentially this
170 phenomenon, where the value of a recently presented item influences the value assigned to a subsequent item. The anchoring effect can be used to demonstrate inconsistencies in decision making, but it can also be used to manipulate potential buyers, and is routinely used in marketing and sales. A vast literature on heuristics and biases catalogues such phenomena (e.g., Gigerenzer et al 1999; Kahneman 2003; Kahneman & Tversky 1984). Future work could examine the structure of these heuristics to determine whether they can be characterized by contingent rules calibration systems of this kind.
5.6. STRATEGIC INTERACTIONS AND CONTINGENT SIGNALING The natural world is rife with situations in which organism can affect one another’s fitness payoffs. These situations can be characterized by mutual interests, conflicting interests or a combination of both. Game theory has provided a valuable formal framework for conceptualizing and investigating cooperation and competition in the natural world (Maynard Smith 1982; Maynard Smith & Price 1973). Typically game theory focuses on the structural features of the interaction, including the payoffs for alternative strategies, the relationships between these payoffs, and the optimal behaviors for each player given the payoff structure. Game theoretic approaches to strategic behavior assume ‘rationality’ which often entails the presupposition that individuals will behave in a way that optimizes payoffs. However, if the payoffs for interactions are dependent on the behavior of others, and the behavior of others can be affected by a variety of factors, the calculation of payoff maximizing option becomes more complex. If organisms have limited cognitive
171 capacities, limited information about the world and the limited abilities to determine the likely outcomes from transmitting various signals that could modify the behavior of interaction partners, payoff maximization is less likely. In general, cooperative and deceptive signaling can play an important role in the instantiation of interdependent payoffs (Hasson 1994; Maynard Smith & Harper 2004), making the maximization of payoffs a strategically complex prospect. The present approach shifts the focus from payoff maximization and rationality to the more computational aspects of interactions, carefully considering the input and output conditions of signaling systems and how organisms can transmit information that does or does not preserve the statistical relationships for which contingent rules have evolved. Deception involves the alteration of another organism’s behavior through signals that violates the statistical relationship (correlation between signal and a beneficial outcome for the receiver) for which the input conditions of the information processing system were selected. This can manifest as the evolution of fixed traits, as is likely to be the case in many changes to the physical phenotype (as in mimicry of wing patterns by butterflies or species typical eggs by cuckoos) or it can manifest behaviorally. Behavioral mimicry obtains when signals are emitted whose purpose is to manipulate the behavior of the receiver for the benefit of the sender and at the expense of the receiver. This section discusses several different categories of strategic behavior, focusing on the input conditions, decision rules and behavioral outputs of the players.
172 5.6.1. Fixed Signals Traits such as coloration and other aspects of phenotype can be stable over the lifetime of an organism. To the extent that these characteristics covaried with fitness because of effect on the behavior of other organisms (through possessing features that met the input conditions of others’ contingent decision making systems), they can be considered signals. For example, the anglerfish’s worm-like appendage covaried with fitness because it met the input conditions of the contingent movement system of prey, inducing them to approach. Most of the features addressed in the previous section can be considered fixed traits.
5.6.2. Channel Contingent Signaling The ability to use information about the channel quality as input to contingent signaling rules can covary with fitness because the use of such contingent rules can make the signaling more effective and/or less costly than it would otherwise be. Ecological features (such as lighting conditions) can contain information (i.e., reduce the statistical uncertainty) about the likelihood that the signal will meet the input conditions of the receiver. There are two main ways that this can happen, 1) environmental conditions promote/degrade the fidelity of signals through amplification/interference and 2) environmental conditions can affect the background noise coming in to the receiver’s information processing systems. To the extent that the sender can make the transmission of the signal contingent on these features in a way that makes the signal more likely to enter the receiver’s input conditions, covariance between the contingent signal transmission rule and fitness will result. If ecological features affect the likelihood of the
173 signal meeting the receiver’s input conditions, senders can benefit from making the transmission of the signal contingent on those ecological features. Both physical features and behaviors that act as signals can be contingent on ecological features. Consider the angler fish again: let us speculate that the angler wiggles the worm-like appendage differently in different ecological conditions, contingently wiggling the appendage more when lighting conditions are more effective for luring prey. This would result in covariance between fitness and the contingent movement rule of the predator (to wiggle with certain light input) because of the effect this had on the contingent movement rule of the prey. Because the ability of the prey to detect the wiggling appendage of the predator can be effected by ecological conditions, it benefits the predator to make the emission of this signal dependent on those ecological conditions. Similarly, an individual emitting a costly signal of mate value (S [IMV]) could benefit from emitting that signal only when the environment promoted signal fidelity and had low levels of background noise (Figure 5.7a).
174
Figure 5.7. a) The emission of a costly signal of mate value (S [IMV]) can be most effective at reaching the receiver if it is given when the environment promotes signal fidelity and does not contain high levels of background noise. This makes it more likely that the signal will reach the receiver and have the opportunity to enter the receiver’s processing system, meeting the threshold (IMV> TMV) and causing the
175 receiver to approach, for example. If the sender makes the emission of a costly signal of mate value (S [IMV]) contingent on high signal fidelity and low noise in the channel (IC > TC), this can reduce costs for the sender, potentially promoting fitness. b) Similarly, the emission of a signal will be more effective if the receiver’s input channels are aligned with the signal. If the sender makes the emission of a costly signal (emit S [IMV]) contingent on the alignment of the receiver’s information channels (IC > TC), this can reduce costs for the sender, potentially promoting fitness. c) The final case illustrated here is of a receiver who emits a signal of channel alignment (S [IC]) which induces a sender to emit a mating display (S [IMV]).
Above it was noted that features of the environment associated with signal fidelity and noise can affect the likelihood that the signal will meet a receiver’s input conditions. However, characteristics of the receiver can also importantly affect the likelihood that the signal will meet input conditions. For example, the alignment of a receiver’s perceptual systems (e.g., through body orientation, eye gaze) can provide information about the likelihood that a signal emitted by the sender will have the opportunity to reach the receiver. To the extent that a sender can use such information about the receiver’s state (including but not limited to the alignment of input channels) to condition signal emission, this can make the signals more targeted and effective, leading to positive covariance of the contingent signal emission rule and fitness. For example, complex and energetically costly mating displays (S [I MV]), would be most effectively targeted towards potential mates whose signal processing channels are directed towards the sender (Figure 5.7b). If the sender’s transmission of information results in positive covariance for the receiver, the receiver can then benefit from emitting signals of alignment of their perceptual channels with the sender. In other words, the transmission of signals of input
176 channel alignment can eventually result in positive fitness covariance for the ‘receiver’ because of the way that it affects the sender. In the mating context, a potential mate might selectively orient (signaling alignment of input channels) to the sender in order to induce that sender to emit a mating display. This is an additional level of contingent signaling, where a receiver can emit signals (e.g., demonstrating alignment of signal processing channels (S [I C]) in order to induce the sender to emit, for instance, a mating display (S [I MV]). This situation is illustrated in Figure 5.7c.
5.6.3. Resource Availability Contingent Signaling Senders might benefit from contingent signaling based on ecological features when the costs and benefits associated with emitting the signal is dependent on the availability of resources. For example, the benefit associated with emitting a signal that induces competitors to move away is likely to be contingent on the value of the resources that might be in contention. In other words, there is more to gain from emitting signal of threat (S [I TH]) to a competitor when the present environment has more highly valued resources (I TA). In this case the signal emission would be contingent on the value of the available resources (Figure 5.8).
177
Figure 5.8. If a signalers can benefit from reducing the spatial proximity of competitors to a valued resource, the ability to make costly signal emission (S [I TH]) contingent on the value of the resource (ITA> TTA), could have positive effects on the signaler’s fitness by making the signaler more likely to emit the signal when there is much to gain from doing so.
The covariance of these various abilities with fitness will be affected by the characteristics of the individuals with whom the individual is signaling (e.g., is the partner emitting honest or deceptive signals). This approach can also be extended to consider contexts with multiple embedded contingent signaling systems where the ability to emit signals, process signals and emit signals about the likelihood that signals will be (or were) processed can each have different covariance with fitness. The next section discusses these extensions of the framework.
178
5.7. EMBEDDED CONTINGENT RULES If changes in an organism’s payoffs are the result of environmental or social contingencies, the ability to generate cognitive representations of these contingencies could provide fitness advantages. If an organism can simulate the likely effects of various behaviors, they can spend less time varying behavioral output in the ‘real world’ (which can be costly), instead, running these potential behavioral outputs through the represented contingent rule before producing them. The ability to generate and use internal representations of environmental and social contingencies can enable an organism to extrapolate based on a small amount of information about the regularities and contingencies in the physical and social world, leading to fewer ‘errors.’ If an organism can contingently emit behaviors based on the results of simulated social and environmental contingencies, this can enable both more effective use of resources and strategic social behavior (Tooby & Cosmides 1992; Tooby & DeVore 1987).
5.7.1. Embedded movement rule representation The section on threshold calibration described rules that might underlie the ability to calibrate movement, benefit emission or signaling thresholds based on feedback from the environment by a sort of ‘trial and error’ process. This process involves continuing to emit outputs that result in positive outcomes, otherwise varying the output. If the consequences of an organism’s behaviors are relatively stable over time, this strategy can allow an organism to narrow in on behavioral outputs that provide benefits and then continue to benefit from these behaviors over the long run. However, if the results of an
179 organism’s behaviors change over time, the calibration rule described above would lead an organism to spend more time varying behavior (which may be costly) and less time repeating behaviors. In other words, if the behavioral landscape of payoffs for behaviors is changing over time, an organism using the calibration strategy will spend more time ‘searching’ for peaks and less time staying on previously discovered peaks. An organism could represent an environmental contingency, ‘checking’ if a potential behavior (M) will result in a high enough beneficial output from the physical environment (Figure 5.9a). These rules in combination would allow an organism to ‘test’ potential behaviors against simulated contingencies before actually emitting them. The ability to represent inputs and outputs of contingencies is denoted by embedding these inputs and outputs inside brackets preceded by an R (for representation). For example, the input of the contingent rule in Figure 5.9a is denoted R [M], and the output is denoted R [I TA].
180
Figure 5.9. a) The ability to simulate potential movement outputs and their consequences can be advantageous because it can allow an organism to select highly beneficial movement outputs. An organism can simulate the effects of various potential movement outputs, emitting a behavior (M) if the output of the simulation meets the threshold (R [ITA]> TTA). b) The ability to simulate potential levels of benefit emission and their consequences for reproductive success can allow an organism to select a level benefit emission that will have optimal effects on kin in the long term. An organism can simulate the results of alternative levels of benefit emission, choosing a level of benefits (B) that results in high enough (R [ISA]> TSA simulated returns to reproductive fitness.
5.7.2. Embedded benefit transmission rule representation The ability to model the contingencies used by other social actors could allow the organism to find the likely optimal level of benefit transmission without undergoing a potentially costly (and possibly impossible) trial and error process. If the organism can
181 effectively model the outcomes of various levels of benefit transmission to kin (e.g., offspring), it can potentially settle on a level of investment that provides high marginal benefits to reproductive success considering the tradeoffs between present and future parental investment opportunities (e.g., Kaplan & Gangestad 2005; Trivers 1972). The ability to represent alternative outcomes of various levels of investment in kin is illustrated in Figure 5.9b.
5.7.3. Embedded mental state representation The ability to model the mental states of others can provide important advantages to individuals living in a social world with strategic others. Indeed, it has been proposed that human evolution has been largely shaped by the need to track expanding social worlds and engage in complex social reasoning (Byrne & Whiten 1998; Dunbar 1993; Whiten & Byrne 1997). It has also been suggested that a deficit in the capacity to simulate the mental states of others (the inability to engage in ‘theory of mind’) might be the central feature of autism (Baron-Cohen 1995). Information about the mental states of others is denoted with the letters TOM (for Theory of Mind) preceding brackets. The value of mental state modeling abilities can be considered in the context of the resource conflict situation discussed earlier. If the competitor for the resource values the resource highly, they will be less likely to move in response to a threatening signal. The ability to consider the competitor’s valuation of the resource in deciding the proper intensity of the signal could be advantageous (Figure 5.10). In such a situation, organisms can benefit from more accurate predictions of a competitor’s behavior. If, for example, the competitor values a contested resource highly (their ITA is high), a more
182 costly signal might be necessary to induce the competitor to move. If the sender can emit a signal that is just strong enough to induce a competitor to move, the sender will be less likely to incur unnecessary costs emitting signals that would be below the receiver’s threshold or waste resource by signaling more strongly than necessary.
Figure 5.10. The ability to simulate the contingent rules of (and information available to) other social entities can be advantageous because it can allow organisms to predict potential behaviors of others. An organism can simulate the effects of signaling to a receiver given the likely rules and representations in the receiver’s information processing system, emitting a threat signal (ITH) if the output of the receiver’s simulated mental operations meets the threshold (TOM [M] > TM).
Many other embedded contingent rules could have provided advantages to organisms dealing with environmental and social contingencies such as those relating to the acquisition of maximum benefits from the physical environment or social partners. Additionally, the representation of the contingent signal emission rules of others can aid
183 an organism in optimally inducing others to emit (or fail to emit) signals that can have fitness consequences. Broadly speaking, these embedded contingent rules can provide strategic advantages for individuals trying to maximize payoffs in a world with complex environmental contingencies and other social actors whose behavior might not be predictable solely on the basis of past behavior. If the ability to use different embedded contingent rules in different domains was advantageous, the proper encapsulation of these contingent rules would be essential for their effecting functioning, especially for modeling the contingent rules and available information inputs of other social actors. Similar ideas have been discussed extensively elsewhere (Cosmides & Tooby 2000a).
5.8. DISCUSSION It is widely appreciated that humans are generalists, having invaded many different ecological and geographic niches. In this section it is suggested that the abilities discussed in earlier sections might have played important roles in this process. Consider, for example, the human capacity to represent and respond to complex information and contingencies about the world. This ability might have enabled humans to consume a wide variety of plant species by making it possible for humans to break down biochemical barriers of a wide variety of plant species spread across the phylogentic tree (Şerban et al 2008) more quickly than the plants could evolve new barriers. This ability to stay ‘one step ahead’ of plant and animal counter-adaptations to consumption through the representation of potential outcomes of alternatives has been called the ‘cognitive
184 niche’ (Cosmides & Tooby 2000b; Tooby & DeVore 1987). These authors have suggested that the ability to model such contingencies enabled humans to succeed in diverse ecological and social environments with widely varying contingencies. The ability to transmit information has also been named as a central feature enabling humans to occupy diverse ecological niches. According to this view, the transmission of cultural information allows the rapid adaptation to changing environmental conditions and the ability to quickly new adopt technologies (Richerson & Boyd 2005). Both the ability to represent contingent relationship and the ability to transmit information might have played important roles in humans’ evolutionary trajectory. Below it is proposed that much of what has been defined as culture can be formalized and discussed in the language of contingent rules, signaling, and representation. This approach could potentially provide formal tools to bridge the divide between ‘genetic’ and ‘cultural’ approaches to understanding human evolutionary history. Interestingly, several components of contingent behavior discussed in this chapter map onto the idea of ‘evoked culture’ suggested by Tooby and Cosmides (1992). In their own words “People living in the same location are likely to experience somewhat similar circumstances, which should evoke the same kind of response from each individual [because of universally shared information processing mechanisms]; people living in different locations are likely to experience somewhat different circumstances, which should evoke different responses from each individual” (p. 210). In other words, if individuals share contingent rules, they can be expected to respond similarly to
185 environmental inputs, resulting in behavioral variation in different environments that is due solely to the environmental inputs (rather than the transmission of information). Tooby and Cosmides also note that there is a class of ‘cultural’ behaviors that are the result of information being transferred from one social entity to another. This transmitted or “reconstructed” culture is defined as “…those representations or regulatory elements that exist originally in at least one mind that come to exist in other minds because observation and interaction between the source and the observer cause inferential mechanisms in the observer to recreate the representations or regulatory elements in his or her own psychological architecture” (Tooby & Cosmides 1992, p. 118). Culture has similarly been defined by Richerson and Boyd as “information capable of affecting individuals’ behavior that they acquire from other members of their species through teaching, imitation and other forms of social transmission” (2005, p. 5). Below a definition of culture is proposed that is consistent with both of these approaches and joins together these ideas with concepts discussed throughout this chapter. This chapter addressed contingent behavior, suggesting that the ability to take in information from the world and use it to condition behavioral outputs can positively covary with fitness. It is noted that the ability to modify components of these rules (thresholds, outputs) and even represent new contingencies can provide additional advantages. Lastly, the ability to model the information and/or contingent rules in another organism’s mind was discussed. It is suggested here that cultural transmission consists of the transfer of any of these components from one mind to another. More specifically, culture can be considered to have the following features:
186 1.
It is characterized by the transmission of information about the world, including temporal assortment information (ITA) and spatial assortment information (ISA), and signals about each of these kinds of information (S [ITA] and S [ISA]) that can meet input conditions of a receiver’s decision rules.
2.
It can transmit information about contingent rules including calibrated thresholds (TTA, TSA), calibrated outputs (M or B levels), or a representation of an entire contingent rule.
3.
It can transmit information about the mental representations of other entities with regard to the information in components 1, 2 and 3.
In this view, the simpler forms of cultural transmission involve the transmission of spatial or temporal assortment information from one entity to another, or the transmission of information about signals emitted by other entities (1). More complex forms of cultural transmission can be though to involve the transmission of information about contingent rules themselves, allowing organisms to share with each other the outcomes of ‘trial and error’ learning (in the case of threshold and output calibration) as well as the outcomes of internal simulations of potential outcomes from behavioral alternatives (from representation of environmental and social contingencies). Perhaps most interestingly, a final component of this framework for conceptualizing cultural transmission is the transfer of information about the mental state representations of other entities (i.e., TOM information) with regard to all of the above features (3).
187
Figure 5.11. It is suggested that cultural transmission entails the recreation of the sender’s information about assortment, signals, previously calibrated threshold, calibrated outputs, embedded contingent rules and embedded mental state information in the brains of receivers. This might occur through passive signaling (receiver observation and imitation of the sender’s behaviors i.e., M or B, that result in beneficial outcomes) or through active signal transmission S [I].
This framework connects diverse ideas to the concept of culture, including those underlying information processing, learning, signaling and representation. It also suggests a framework in which to consider neutral, cooperative and manipulative cultural transmission, considering the fitness covariance of participating senders and receivers. This approach might provide unique tools to help understand the factors shaping the susceptibility of receivers to potentially manipulative cultural transmission such as those underlying cults, superstition, obedience, and possibly even gossip. Another area that can be considered from this perspective is that of social and technological leadership. The
188 ability to effectively transmit cultural information is likely to be an important component of successful leadership in these domains, while effective following requires the ability to adopt and integrate such information from others. The literature on prestige biased transmission has addressed similar issues, but with a focus on social exchange, i.e., the exchange of valuable information for socially conferred status (Henrich & GilWhite 2001). The framework and definition of culture laid out here suggests interesting connections among several theories and approaches in Psychology, Biology and Anthropology. Further, it provide a more computationally specified definition of culture than that suggested by Richerson and Boyd (2005) and Tooby and Cosmides (1992). This approach to culture and social dynamics can guide future conceptual work and future modeling efforts.
5.9. CONCLUSION Chapter 1 of this thesis offers an extensive treatment of the idea that organism can benefit from contingent rules which operate on information about spatial and temporal assortment to beneficially emit benefits and move through space, respectively. In this chapter it is suggested that signaling, or the transmission of information from one entity to another, began as the ability of one organism to meet the input conditions of another organism’s contingent movement and benefit transmission rules. Diverse information from the physical and social environment can meet the input conditions of these rules, leading the receiver to behave in ways that can be beneficial or harmful. The evolution of
189 honest and deceptive signaling is discussed in this context, with the suggestion that covariance of fitness with signal processing for the receiver is the key factor in the emergence and evolution of honest/cooperative signaling. The ability of organisms to use flexible contingent rules is also discussed, in particular the ability to calibrate thresholds and output based on feedback from the environment. Finally, it is suggested that embedded contingent rules or representations of social and environmental contingencies can be beneficial for organism occupying complex physical and social niches. In the discussion, speculations are offered about the transmission of culture. A definition of culture is proposed that relies on the ability to transmit information about 1) the world, 2) contingent rules and 3) representations of the content of other entities minds with regard to 1, 2 and 3. This chapter speculates that complex information processing abilities can be built from simple rules for contingent movement and contingent cooperation. These rules are formalized and computationally specified, allowing for them to be modeled in future agent-based simulations such as the SIMPLE model described in the first chapter. Future work can also address conceptual issues in greater depth, including a more thorough treatment of signal detection theory; in particular, its applications to the issues surrounding fitness covariance in honest vs. deceptive signaling and agent-environment interaction in signal transmission and processing. Decision rules underlying contingent behavior can underlie both simple and complex manifestations of cooperation. Contingent movement and benefit transmission rules can result in complex phenomena such as those underlying cooperative communication, deceptive signaling, and parental investment. Further, various aspects of
190 learning/reinforcement, the representing of contingencies and mental states, and possibly even the processing of cultural information might rely on the ability to modify and transmit various components of these contingent movement and benefit transmission rules. The framework outlined in this chapter demonstrates that slowly adding layers of complexity to initially simple rules can yield conceptual transparency and afford the opportunity to explore connections among diverse and seemingly disconnected biological and social phenomena. It is suggested here that contingent movement and contingent benefit transmission form the foundation for cognition and social interaction across the phylogenetic tree, from the simplest organisms to the most computationally sophisticated, and that the capacity for contingent behavior plays an important role in phenomena as diverse as mating displays, brood parasitism, fruit coloration, resource conflict and, of course, cooperation in its varied shapes and forms.
191
APPENDIX A SIMPLE (Simulation of agent Interaction through Movement and Production in a Local Environment) was developed using NetLogo 4.0 (Wilensky 1999). This Chapter 1 model is a 1-dimensional cellular automaton model with a row of agents occupying the cells. As time moves forward and the active row is moved down, a record of the energy in the environment and the location of agents is maintained so that temporal and spatial changes can be visually discerned (see Figures 1.2 and 1.3). Initially, agents are place on the active row with a .5 probability of being Producers (blue) or Scroungers (red). The patches that make up the environment are endowed with an initial energy level of 1.5, and agents can consume and produce energy on these patches to change the energy level of the local environment (higher energy levels are represented by lighter pink/white and lower energy levels are represented by darker pink/black). During each time period, the environment, population and agents change according to the specified rules. Energy diffuses in the environment according to the diffusion parameter (.5 of the energy diffuses to neighboring patches each time period). The following commands are executed by agents: 1) Production: .55 units of energy is produced and either placed in environment (Producers) or stored in the agent (Scroungers) 2) Movement: agents move according to selected rules 3) Consumption: use .5 units of energy from environment (use stored energy if environment is depleted)
192 4) Storage tax: stored energy is taxed at a fixed rate of .1 unit of energy Changes in the population occur when the following commands are executed: 1) Death: agents that have depleted local environment and energy stores die 2) Reproduction: if there is a free patch ahead or behind, create a copy on that patch and split energy with that copy (only in Figure 1.2d)
Movement rules used by agents are discussed in the description of the SIMPLE model in the body of Chapter 1. These include no movement, directional movement, the Walk Away rule and a Benefit Approach rule.
193
APPENDIX B In this Chapter 4 model, agents were given an initial randomly determined energy level between 1 and 25. They were eliminated from the population if their energy level reached 0, and reproduced if their energy reached 50. Reproduction involved the splitting of the agent’s energy with a newly created clone offspring (mutating I and T with some probability). That offspring was then placed on the same patch, and therefore started in the same group as the parent. In this model, the number of agents in each simulation was 100, the density level approximately 3 patches per agent, the multiplier 2, and the mutation rate 5%. The spatial lattice inhabited by these agents is made up of ‘patches,’ or unique locations, arranged in a torroid.
194
BIBLOGRAPHY Aktipis CA. 2000. The effect of behavioral assortment on selection dynamics: externalities, information processing and subjective commitment. Proceedings of the Artificial Life VII Workshops Aktipis CA. 2004. Know when to walk away: contingent movement and the evolution of cooperation. Journal of Theoretical Biology 231:249-60 Aktipis CA. 2006. Recognition memory and the evolution of cooperation: How simple strategies succeed in an agent-based world. Adaptive Behavior 14:239-47 Aktipis CA. manuscript. The evolution of cooperation in groups via a simple "Walk Away" strategy. Ashlock D, Smucker MD, Stanley EA, Tesfatsion L. 1996. Preferential partner selection in an evolutionary study of Prisoner's Dilemma. BioSystems 37:99-125 Aviles L. 1993. Interdemic selection and the sex ratio: a social spider perspective. The American Naturalist 142:320-45 Aviles L. 1998. Cooperation, nonlinear dynamics and the levels of selection. presented at The Second International Conference on Complex Systems. Nashua, NN Avilés L. 2002. Solving the freeloaders paradox: genetic associations and frequencydependent selection in the evolution of cooperation among nonrelatives. PNAS 99:14268-73 Avilés L, Fletcher JA, Cutter A. 2004. The kin composition of groups: trading group size for degree of altruism. The American Naturalist 164:132-44 Axelrod R. 1984. The Evolution of Cooperation. New York: Basic Books
195 Axelrod R. 1997. The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton: Princeton University Press Axelrod R, Hamilton WD. 1981. The Evolution of Cooperation. Science, New Series 211:1390-6 Barclay P, Willer R. 2007. Partner choice creates competitive altruism in humans. Proceedings of the National Academy of Sciences of the United States of America: B Biological Sciences 274:749-53 Baron-Cohen S. 1995. Mindblindness: An Essay on Autism and Theory of Mind. Cambridge, MA: MIT Press Bates HW. 1863. The naturalist on the river Amazons. London: Murray Bergmuller R, Johnstone RA, Russell AF, Bshary R. 2007a. Integrating cooperative breeding into theoretical concepts of cooperation. Behavioural Processes 76:6172 Bergmuller R, Russell AF, Johnstone RA, Bshary R. 2007b. On the further integration of cooperative breeding and cooperation theory. Behavioural Processes 76:170-81 Bergstrom CT, Lachmann M. 1997. Signaling among relatives. I. Is costly signaling too costly? C. T. Bergstrom and M. Lachmann. Philos Trans R Soc Lond B Biol Sci 352:609-17 Boone RT, Macy MW. 1999. Unlocking the doors to prisoner's dilemma: dependence, selectivity, and cooperation. Social Psychology Quarterly 62:32-52 Boorman SA, Levit PR. 1980. The Genetics of Altrusim. New York: Academic Press Bowles S, Gintis H. 2004. The evolution of strong reciprocity: cooperation in heterogeneous populations. Theoretical Population Biology 65:17-28
196 Boyd R, Richerson PJ. 1985. Culture and the evolutionary process. Chicago: University of Chicago Press Boyd R, Richerson PJ. 1988. The evolution of reciprocity in sizable groups. Journal of Theoretical Biology 132:337-56 Boyd R, Richerson PJ. 1992. Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethology and sociobiology 13:171-95 Brandt H, Hauert C, Sigmund K. 2006. Punishing and abstaining for public goods. Proceedings of the National Academy of Sciences of the United States of America 103:495-7 Brauchli K, Killingback T, Doebeli M. 1999. Evolution of cooperation in spatially structured populations. Journal of Theoretical Biology 200:405-17 Brown RA, Armelagos GJ. 2001. Apportionment of racial diversity: a review. Evolutionary Anthropology 10:34-40 Bull JJ, Rice WR. 1991. Distinguishing mechanisms for the evolution of co-operation. Journal of Theoretical Biology 149:63-74 Buston PM, Balshine S. 2007. Cooperating in the face of uncertainty: A consistent framework for understanding the evolution of cooperation. Behavioural Processes 76:152-9 Byrne R, Whiten A. 1998. Machiavellian Intelligence: Social Expertise and The Evolution of Intellect in Monkeys, Apes and Humans. Oxford: Clarendon Press Chisholm JS. 1996. The evolutionary ecology of attachment organization. Human Nature 7:1-38
197 Connor RC. 1992. Egg-trading in simultaneous hermaphrodites: an alternative to Titfor-Tat. Journal of Evolutionary Biology 5:523-8 Cook RM, Hubbard SF. 1977. Adaptive Searching Strategies in Insect Parasites. The Journal of Animal Ecology 46:115-25 Cosmides L, Tooby J. 1992. Cognitive adaptations for social exchange. In The Adapted Mind, ed. JH Barkow, L Cosmides, J Tooby, pp. 163-228. New York: Oxford University Press Cosmides L, Tooby J. 2000a. Consider the source: the evolution of adaptations for decoupling and metarepresenation. In Metarepresentations: A multidisciplinary perspective, ed. D Sperber. New York: Oxford University Press Cosmides L, Tooby J. 2000b. Consider the source: the evolution of adaptations for decoupling and metarepresentation. In Metarepresentations: A multidisciplinary Perspective, ed. D Sperber. NY: Oxford Cox SJ, Sluckin TJ, Steele J. 1999. Group size, memory and the interaction rate in the evolution of cooperation. Current Anthropologist 40:369-77 Dawes RM. 1980. Social dilemmas. Annual Review of Psychology 31:169-93 Dawkins R. 1976/1989. The Selfish Gene. Oxford: Oxford University Press Dawkins R. 2008. The Group Delusion. New Scientist Magazine:17 deVos H, Zeggelink E. 1994. Reciprocal altruism in human social evolution: the viability of altruism with a preference for "old-helping-partners". Evolution and Human Behavior 18:261-78 Doebeli M, Hauert C, Van Baalen M. 2005. Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game. Ecology Letters 8:748-66
198 Dugatkin LA. 1992. The evolution of the "con artist". Ethology and Sociobiology 13:3-18 Dugatkin LA, Wilson DS. 1991. ROVER: A strategy for exploiting cooperators in a patchy environment. The American Naturalist 138:687-701 Dugatkin LA, Wilson DS. 1992. Rover: A strategy for exploiting cooperators in a patchy environment. The American Naturalist 138:687-701 Dunbar RIM. 1993. Coevolution of neocortical size, group size and language in humans. Behavioral and Brain Sciences 16:681-735 Enquist M, Leimar O. 1993. The evolution of cooperation in mobile organisms. Animal Behavior 45:747-57 Epstein JM. 1998. Zones of cooperation in a demographic prisoner's dilemma. Complexity 4:36-48 Eshel I, Cavalli-Sforza LL. 1982. Assortment of encounters and evolution of cooperativeness. Proceeding of the National Academy of Science 79:1331-5 Fehr E, Fischbacher U. 2004. Social norms and human cooperation. Trends in Cognitive Science 8:185-90 Fehr E, Fischebacher U, Gachter S. 2002. Strong reciprocity, human cooperation, and the enforcement of social norms. Human Nature 13:1-25 Fehr E, Fishchbacher U. 2003. The nature of human altruism. Nature 425:785-91 Fehr E, Gachter S. 2002. Altruistic punishment in humans. Nature 415:137-40 Ferriere R, Michod RE. 1995. Invading wave of cooperation in a spatial iterated prisoner's dilemma. Proceedings of the Royal Society, London: Series B, Biological Sciences 259:77-83
199 Ferriere R, Michod RE. 1996. The Evolution of Cooperation in Spatially Heterogeneous Populations. The American Naturalist 147:692-717 Fletcher JA, Zwick M. 2001. Altruism, the prisoner's dilemma and the components of selection. Proceedings of the IEEE Systems, Man and Cybernetics Conference, pp. 1966-71 Fletcher JA, Zwick M. 2004. Strong Altruism Can Evolve in Randomly Formed Groups. Journal of Theoretical Biology 228:303-13 Fletcher JA, Zwick M. 2007. The evolution of altruism: game theory in multilevel selection and inclusive fitness. Journal of Theoretical Biology 245:26-36 Fodor J. 1983. The Modularity of Mind. Cambridge, MA: MIT Press Fowler JH. 2005. Altruistic punishment and the origin of cooperation. Proceedings of the National Academy of Sciences of the United States of America 102:7047-9 Frank RH. 1988. Passions Within Reason: The strategic role of the emotions. New York: W.W. Norton & Company, Inc. 304 pp. Friedman JW, Hammerstein P. 1991. To trade or not to trade; that is the question. In Game Equilibrium Models I, ed. R Selten, pp. 257-75. Berlin: Springer-Verlag Fu F, Chen X, Liu L, Wang L. 2007. Promotion of cooperation induced by the interplay between structure and game dynamics. Physica A 383:651-9 Gallistel CR. 2000. The replacement of general-purpose learning models with adaptively specialized learning modules. In The cognitive neurosciences, ed. MS Gazzaniga, pp. 1179-91. Cambridge, MA: MIT Press
200 Gallistel CR. 2002. The principle of adaptive specialization as it applies to learning and memory. In Principles of human learning and memory, ed. RH Kluwe, G Lüer, F Rösler, pp. 250-80. Birkenaeuser: Basel Gallistel CR. 2003. Conditioning from an information processing perspective. Behavioural Processes 62:12 Gallistel CR, Gibbon J. 2000. Time, rate and conditioning. Psychological Review 107:289-344 Gardner A, Westt SA. 2004. Cooperation and Punishment, Especially in Humans. American Naturalist 164:753-64 Gigerenzer G. 2000. Adaptive thinking: Rationality in the real world. New York: Oxford University Press Gigerenzer G, Todd PM, The ABC Research Group. 1999. Simple heuristics that make us smart. New York: Oxford University Press Gintis H. 2000. Strong reciprocity and human sociality. Journal of Theoretical Biology 206:169-79 Gintis H. 2003. The hitchhiker's guide to altruism: gene-culture coevolution, and the internalization of norms. The Journal of Theoretical Biology 220:407-18 Green DM, Swets JA. 1966. Signal Detection Theory and Psychophysics. New York: Wiley Gubernick DG. 1981. Parent and infant attachment in mammals. In Parental care in mammals, ed. DG Gubernick Haig D. 2002. Genomic Imprinting and Kinship: Rutgers University Press
201 Hamilton IM, Taborsky M. 2005. Contingent movement and cooperation evolve under generalized reciprocity. Proceedings of the Royal Society, B 272:2259-67 Hamilton WD. 1964a. The genetical evolution of social behavior I. J. Theor. Biol. 7:1-16 Hamilton WD. 1964b. The geneticial evolution of social behavior II. J. Thoer. Biol. 7:1752 Hamilton WD. 1975. Innate social aptitudes of man: an approach from evolutionary genetics. In Biosocial anthropology, ed. R Fox, pp. 133-55. London: Malaby Hammerstein P. 2003. Why is reciprocity so rare in social animals? a protestant appeal. In Genetic and cultural evolution of cooperation, ed. P Hammerstein, pp. 83-93. United States of America: MIT and Freie Universität Berlin Hammond RA, Axelrod R. 2006. Evolution of contingent altruism when cooperation is expensive. Theoretical Population Biology 69:333-8 Hardin G. 1968. The Tragedy of the Commons. Science 162:1243-8 Hasson O. 1994. Cheating Signals. Journal of Theoretical Biology 167:223-38 Hauert C. 2002. Effects of space in 2 x 2 games. International Journal of Bifurcation and Chaos 12:1531-48 Hauert C, Doebeli M. 2004. Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428:643-6 Hauert C, Holmes M, Doebeli M. 2006. Evolutionary games and population dynamics: maintenance of cooperation in public goods games. Proceedings: Biological Sciences 273:2565-70 Hauk E. 1999. Multiple prisoner's dilemma games with(out) an outside option: an experimental study. Universitat Pompeu Fabra, Barcelona, Spain
202 Hauser D. 1996. The Evolution of Communication. Cambridge, MA: MIT Press Henrich J, Gil-White F. 2001. The Evolution of Prestige: freely conferred deference as a mechanism for enhancing the benefits of cultural transmission. Evolution and Human Behavior 22:16596 Ifti M, Killingback T, Doebeli M. 2004. Effects of neighbourhood size and connectivity on the spatial Continuous Prisoner's Dilemma. Journal of Theoretical Biology 231:97-106 Janssen MA, Goldstone RL. 2006. Dynamic-persistence of cooperation in public good games when group size is dynamic. Journal of Theoretical Biology 243:134-42 Johnsgard PA. 1997. The Avian Brood Parasites: Deception at the Nest. USA: Oxford University Press Johnstone R, Grafen A. 1992. The continuous Sir Philip Sidney game: a simple model of biological signaling. Journal of Theoretical Biology 156:215-34 Kahneman D. 2003. Maps of bounded rationality: psychology for behavioral economics. American Economic Review:1449-75 Kahneman D, Tversky A. 1984. Choices, Values and Frames. American Psychologist 39:341-50 Kaplan H, Gangestad S. 2005. Life History Theory and Evolutionary Psychology. In. The Handbook of Evolutionary Psychology. In The Handbook of Evolutionary Psychology, ed. DM Buss, pp. 68-95: John Wiley and Sons Kenrick DT, Li NP, Butner J. 2003. Dynamical evolutionary psychology: individual decision rules and emergent social norms. Psychological Review 110:3-28
203 Kenrick DT, Maner JK, Butner J, Li NP, Becker DV, Schaller M. 2002. Dyanmical evolutionary psychology: mapping the domains of the new interactionist paradigm. Personality and Social Psychology Review 6:247-356 Killingback T, Bieri J, Flatt T. 2006. Evolution in group-structured populations can resolve the tragedy of the commons. Proceedings of the Royal Society B: Biological Sciences 273:1477-81 Killingback T, Doebeli M. 1996. Spatial evolutionary game theory: hawks and doves revisited. Proceedings of the Royal Society, London: Series B, Biological Sciences 263:1135-44 Krebs J, Dawkins R. 1984. Animal signals: mind-reading and manipulation. In Behavioral ecology: an evolutionary approach, ed. J Krebs, N Davies, pp. 38040. Oxford: Blackwell Scientific Lachmann M, Bergstrom C. 1998. Signaling among relatives. II. Beyond the tower of Babel. Theoretical Population Biology 52:146-60 Ledyard JO. 1995. Public goods: a survey of experimental research. In Handbook of Experimental Economics, ed. J Kagel, AE Roth, pp. 111-94. Princeton, NJ: Princeton University Press Lehmann L, Keller L. 2006. The evolution of cooperation and altruism – a general framework and a classification of models. Journal of Evolutionary Biology 19:1365-76 Lehmann L, Keller L, Sumpter DJT. 2007. The evolution of helping and harming on graphs: the return of the inclusive fitness effect. Journal of Evolutionary Biology 20:2284-95
204 Lindgren K, Nordahlb MG. 1994. Evolutionary dynamics of spatial games. Physica D: Nonlinear Phenomena 75:292-309 MacArthur RH, Pianka ER. 1966. On optimal use of a patchy environment. The American Naturalist 100:603-9 MacAurthur RH, Wilson EO. 1967. The theory of island biogeography. Princeton, NJ: Princeton University Press. 203 pp. Marshall JAR, Rowe JE. 2003. Viscous Populations and Their Support for Reciprocal Cooperation. Artificial Life 9:327-35 Masuda N, Aihara K. 2003. Spatial prisoner's dilemma optimally played in small-world networks. Physics Letters A 313:55-62 Maynard Smith J. 1964. Group selection and kin selection. Nature 201:1145-7 Maynard Smith J. 1982. Evolution and the Theory of Games: Cambridge University Press Maynard Smith J. 1991. Honest signaling: the Philip Sidney game. Animal Behaviour 42:1034-5 Maynard Smith J. 1995. Animal signals: models and terminology. Journal of Theoretical Biology 177:305-11 Maynard Smith J, Harper D. 2004. Animal Siganals. USA: Oxford University Press Maynard Smith J, Price GR. 1973. The Logic of Animal Conflict. Nature 246:15-8 Maynard Smith J, Szathmary E. 1995. The Major Transitions in Evolution. Oxford: Oxford University Press. 346 pp. Maynard-Smith J. 1964. Group selection and kin selection. Nature 201:1145-7
205 Mealy L. 1996. Enhanced memory for the faces of cheaters. Ethology and sociobiology 17:119-28 Mendoza SP, Reeder DM, Mason WA. 2002. Nature of proximate mechanisms underlying primate social systems: simplicity and redundancy. Evolutionary Anthroplogy Suppl 1:112-6 Michod RE. 1999. Darwinian Dynamics: Evolutionary Transitions in Fitness and Individuality. Princeton, NJ: Princeton University Press. 262 pp. Milinski M, Semmann D, Krambeck H-J. 2002. Reputation helps solve the 'tragedy of the commons'. Nature 415:424-6 Milinski M, Wedekind C. 1998. Working memory constrains human cooperation in the Prisoner's Dilemma. Proceedings of the National Academy of Science 95:13755-8 MIT Media Laboratory. 2001. StarLogo. Cambridge, MA MIT Media Laboratory. 2003. Starlogo. Cambridge, MA Mitteldorf J, Wilson DS. 2000. Population viscosity and the evolution of altruism. Journal of Theoretical Biology 2004:481-96 Nakamaru M. 2006. Lattice models in ecology and social sciences. Ecological Research:3 Nakamaru M, Kawata M. 2002. Evolution of rumors that discriminate lying defectors. Evolutionary Ecology Research 6:261-83 Nesse RM. 2001. Natural Selection and the Capacity for Subjective Commitment. In Evolution and the Capacity for Commitment, ed. RM Nesse, pp. 1-44. New York, New York: Russel Sage Foundation
206 Noe R, Hammerstein P. 1994. Biological markets: supply and demand determine the effect of partner choice in cooperation, mutualism and mating. Behavioral Ecology and Sociobiology 35:1-11 Nowak M, Sigmund K. 1993. A strategy of win-stay, lose-shift that outperforms tit-fortat in the Prisoner's Dilemma game. Nature 364:56-8 Nowak MA. 2006. Five rules for the evolution of cooperation. Science 314:1560-3 Nowak MA, May RM. 1992. Evolutionary games and spatial chaos. Nature 359:826-9 Nowak MA, Sasaki A, Taylor C, Fudenberg D. 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428:646-50 Nowak MA, Sigmund K. 1998a. Evolution of indirect reciprocity by image scoring. Nature 393:573-7 Nowak MA, Sigmund K. 1998b. The dynamics of indirect reciprocity. Journal of Theoretical Biology 194:561-74 Ohtsuki H, Hauert C, Lieberman E, Nowak MA. 2006. A simple rule for the evolution of cooperation on graphs and social networks. Nature 441:502-5 Ohtsuki H, Nowak MA. 2007. Direct reciprocity on graphs. Journal of Theoretical Biology 247:462-70 Orbell JM, Dawes RM. 1993. Social welfare, cooperators' advantage, and the option of not playing the game. American Sociological Review 58:787-800 Orbell JM, Schwartz-Shea P, Simmons RT. 1984. Do cooperators exit more readily than defectors? The American Political Science Review 78:147-62 Pacheco JM, Santos FC. 2005. Network dependence of the dilemmas Of cooperation. AIP Conference Proceedings 776:90-100
207 Panchanathan K, Boyd R. 2003. A tale of two defectors: the importance of standing for evolution of indirect reciprocity. Journal of Theoretical Biology:115-26 Panchanathan K, Boyd R. 2004. Indirect reciprocity can stabilize cooperation without the second-order free rider problem. Nature 432:499-502 Pasteur G. 1982. A Classification Review of Mimicry Systems. Annual Review of Ecology and Systematics 13:169-99 Peck JR, Feldman MW. 1986. The evolution of helping behavior in large, randomly mixed populations. American Naturalist 127:209-21 Pepper J. 2000. Relatedness in trait group models of social evolution. Journal of Theoretical Biology 206:355-68 Pepper J. 2007. Simple models of assortment through environmental feedback. Artificial Life 13:1-9 Pepper JW, Smuts B. 1999. The evolution of cooperation in an ecological context: an agent-based model. In Dynamics in human and primate societies, ed. T Kohler, G Gumerman, pp. 45-76: Oxford University Press Pepper JW, Smuts BB. 2002. A mechanism for the evolution of altruism among nonkin: Positive assortment through environmental feedback. The American Naturalist 160:205-13 Price GR. 1970. Selection and covariance. Nature 227:520-1 Price ME, Cosmides L, Tooby J. 2002. Punitive sentiment as an anti-free rider psychological device. Evolution and Human Behavior 23:203-31 Queller DC. 1992. Quantitative Genetics, Inclusive Fitness, and Group Selection. The American Naturalist 139:540-58
208 Rescorla RA, Wagner AR. 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II, ed. AH Black, WF Prokasy, pp. 64-99: Appleton-Century-Crofts Richerson P, Boyd R. 1998. The evolution of ultrasociality. In Indoctrinability, ideology and warfare, ed. I Eibl-Eibesfeldt, FK Salter, pp. 71-96. New York: Berghahn Books Richerson PJ, Boyd R. 2005. Not by genes alone: How culture transformed human evolution: University of Chicago Press Sachs JL, Mueller UG, Wilcox TP, Bull JJ. 2004. The evolution of cooperation. The Quarterly Review of Biology 79:135-60 Santos FC, Pacheco JM. 2006. A new route to the evolution of cooperation. Journal of Evolutionary Biology 19:726-33 Santos FC, Pacheco JM, Lenaerts T. 2006. Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proceedings of the National Academy of Sciences of the United States of America 103:3490-4 Santos FC, Rodrigues JF, Pacheco JM. 2005. Graph topology plays a determinant role in the evolution of cooperation. Proceedings: Biological Sciences 273:51-5 Schuessler R. 1989. Exit threats and cooperation under anonymity. Journal of Conflict Resolution 33:728-49 Şerban P, Wilson JRU, Vamosi JC, Richardson DM. 2008. Plant Diversity in the Human Diet: Weak Phylogenetic Signal Indicates Breadth. BioScience 58:151–9 Simpson EH. 1951. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, B 13:238-41
209 Sober E, Wilson DS. 1998. Unto Others: The Evolution and Psychology of Unselfish Behavior. Cambridge, Massachusetts: Harvard University Press. 394 pp. Spector L, Klein J. 2006. Genetic Stability and Territorial Structure Facilitate the Evolution of Tag-Mediated Altruism. Artificial Life 12:553-60 Stephens DW, Krebs JR. 1986. Foraging Theory. Princeton: Princeton University Press Strassmann JE, Queller DC. 2007. Altruism among Amoebas. Natural History 116:24-9 Suzuki R, Arita T. 2003. Evolutionary Analysis on Spatial Locality in N-Person Iterated Prisoner's Dilemma. International Journal of Computational Intelligence & Application 3:177-88 Taylor PD, Day T. 2004. Behavioural evolution: Cooperate with thy neighbour? Nature 428:611-2 Taylor PD, Day T, Wild G. 2007. Evolution of cooperation in a finite homogeneous graph. Nature 447:469-72 Todd PM, Billari FC, Simao J. 2005. Aggregate age-at-marriage patterns from individual mate-search heuristics. Demography 42:559-74 Tooby J, Cosmides L. 1992. The psychological foundations of culture. In The Adapted Mind, ed. JH Barkow, L Cosmides, J Tooby, pp. 19-136. New York: Oxford University Press Tooby J, DeVore I. 1987. The reconstruction of hominid behavioral evolution through strategic modeling. In The evolution of human behavior: Primate models, ed. WG Kinzey. Albany, NY: SUNY Press
210 Traulsen A, Nowak MA. 2006. Evolution of cooperation by multilevel selection. Proceedings of the National Academy of Sciences of the United States of America 103:10952-5 Trivers RL. 1971. The evolution of reciprocal altruism. The Quarterly Review of Biology 46:35-57 Trivers RL. 1972. Parental Investment and Sexual Selection. In Sexual Selection and the Descent of Man 1871-1971, ed. BG Campbell, pp. 136-79. Chicago: Aldine Van Valen L. 1977. The Red Queen. The American Naturalist 111:809-10 Vanberg VJ, Congleton RD. 1992. Rationality, morality and exit. The American Political Science Review 86:418-31 von Neumann J, Morgenstern O. 1944. Theory of games and economic behavior. Princeton, NJ: Princeton University Press Wakano JY. 2007. Evolution of cooperation in spatial public goods games with common resource dynamics. Journal of Theoretical Biology 247:616-22 West SA, Herre EA. 1998. Partial local mate competition and the sex ratio: A study of non-pollinating fig wasps. Journal of Evolutionary Biology 11:531-48 Whiten A, Byrne RW. 1997. Machiavellian Intelligence II. Cambridge: Cambridge University Press Wilensky U. 1999. NetLogo. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University Wiley RH. 1994. Errors, exaggeration, and deception in animal communication. In Behavioral mechanims in ecology, ed. L Real, pp. 157-89. Chicago: University of Chicago Press
211 Williams GC. 1966. Adaptation and Natural Selection. Princeton, NJ: Princeton University Press. 307 pp. Wilson DS. 1977. Structured Demes and the Evolution of Group-Advantageous Traits. The American Naturalist 111:157-85 Wilson DS. 1979. Structured demes and trait-group variation. American Naturalist 113:606-10 Wilson DS. 1983. The group selection controversy: history and current status. Ann. Rev. Ecol. Syst. 14:159-87 Wilson DS. 1987. Altruism in mendelian populations derived from sibling groups: the haystack model revisted. Evolution 41:1059-70 Wilson DS, Dugatkin LA. 1997. Group selection and assortative interactions. The American Naturalist 149:336-51 Wilson DS, Pollock G, Dugatkin LA. 1992. Can altruism evolve in purely viscous populations. Evolutionary Ecology Research 6:331-41 Wilson DS, Wilson EO. 2007. Rethinking the theoretical foundations of sociobiology. The Quarterly Review of Biology 82:327-48 Wright S. 1931. Evolution in Mendelian populations. Genetics 16:97-159 Wright S. 1945. Tempo and mode in evolution: a critical review. Ecology 26:415-9 Wynne-Edwards VC. 1962. Animal dispersion in relation to social behavior. New York: Hafner Publishing Co. Wynne-Edwards VC. 1963. Intergroup selection in the evolution of cooperation. Nature 200:623-6
212 Yamagishi T, Hayashi N. 1996. Selective play: social embeddedness of social dilemmas. In Frontiers in Social Dilemma Research, ed. W Liebrand, D Messick, pp. 363-84. Berlin: Springer Yamagishi T, Hayashi N, Jin N. 1994. Prisoner's dilemma networks: Selection strategy versus action strategy. In Social dilemmas and cooperation, ed. U Schulz, W Albers, U Mueller, pp. 233-50. Berlin: Springer-Verlag Zahavi A. 1977. The cost of honesty (Further remarks on the handicap principle). Journal of Theoretical Biology 67:603-5 Zhang F, Hui C, Han X, Li Z. 2005. Evolution of cooperation in patchy habitat under patch decay and isolation. Ecological Research 20:461-9
