BioSystems 101 (2010) 79–87

Contents lists available at ScienceDirect

BioSystems journal homepage: www.elsevier.com/locate/biosystems

Coevolution of honest signaling and cooperative norms by cultural group selection István Scheuring a,b,∗ a Department of Plant Taxonomy and Ecology, Research Group of Theoretical Biology and Ecology, Eötvös Loránd University and the Hungarian Academy of Sciences, Budapest, H-1117, Pázmány Péter sétány, 1/c, Hungary b Konrad Lorenz Institute, Altenberg, A-3422, Adolf Lorenz Gasse 2, Austria

a r t i c l e

i n f o

Article history: Received 1 March 2010 Received in revised form 16 April 2010 Accepted 27 April 2010 Keywords: Human cooperation Communication Social norm Group selection Indirect altruism

a b s t r a c t Evolution of cooperative norms is studied in a population where individual and group level selection are both in operation. Individuals play indirect reciprocity game within their group and follow second order norms. Individuals are norm-followers, and imitate their successful group mates. Aside from direct observation individuals can be informed about the previous actions and reputations by information transferred by others. A potential donor estimates the reputation of a potential receiver either by her own observation or by the opinion of the majority of others (indirect observation). Following a previous study (Scheuring, 2009) we assume that norms determine only the probabilities of actions, and mutants can differ in these probabilities. Similarly, we assume that individuals follow a stochastic information transfer strategy. The central question is whether cooperative norm and honest social information transfer can emerge in a population where initially only non-cooperative norms were present, and the transferred information was not sufficiently honest. It is shown that evolution can lead to a cooperative state where information transferred in a reliable manner, where generous cooperative strategies are dominant. This cooperative state emerges along a sharp transition of norms. We studied the characteristics of actions and strategies in this transition by classifying the stochastic norms, and found that a series of more and more judging strategies invade each other before the stabilization of the so-called generous judging strategy. Numerical experiments on the coevolution of social parameters (e.g. probability of direct observation and the number of indirect observers) reveal that it is advantageous to lean on indirect observation even if information transfer is much noisier than for direct observation, which is because to follow the majorities’ opinion suppresses information noise meaningfully. © 2010 Elsevier Ireland Ltd. All rights reserved.

1. Introduction Evolutionary origin and stability of indirect reciprocity where the return of altruistic aid is expected from someone other than the recipient of the aid is one that is very characteristic of human nature. This behavior can only be explained if the actions are observed and classified by the members of society with the help of a social norm (Trivers, 1985; Alexander, 1987). Knowing the score (reputation) of a potential recipient (and the donor) and the norm followed by the potential donor, she can decide whether her recipient is worth donating to or not. If free-riders are excluded effectively from the interaction by this norm then indirect reciprocity can be maintained.

∗ Correspondence address: Department of Plant Taxonomy and Ecology, Research Group of Theoretical Biology and Ecology, Eötvös Loránd University and the Hungarian Academy of Sciences, Budapest, H-1117, Pázmány Péter sétány, 1/c, Hungary. E-mail address: [email protected]. 0303-2647/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.biosystems.2010.04.009

By studying this concept within a mathematical framework Nowak and Sigmund (1998a,b) have shown that cooperation by indirect reciprocity is maintained by a norm called image scoring. In their model individuals score increases by one if she donates to a recipient and decreases by one if she refuses donation. Individuals who follow image scoring help only those individuals whose score is above a threshold, so individuals that were altruistic enough in the past are favored. These keystone papers catalyzed a series of studies, including experimental works (e.g. Wedekind and Milinski, 2000; Fehr and Fischbacher, 2003), and a range of analytical and numerical investigations (e.g. Leimar and Hammerstein, 2001; Brandt and Sigmund, 2004; Panchanathan and Boyd, 2003, 2004; Ohtsuki and Iwasa, 2004; Chalub et al., 2006; Pacheco et al., 2006). Leimar and Hammerstein (2001) pointed out that image scoring is not an evolutionary stable strategy if group structure of human populations and inherent decision stochasticity is taken into account. They found that a so-called standing strategy which offers help if its score is below a critical level, can overcome image scoring strategy. Despite

80

I. Scheuring / BioSystems 101 (2010) 79–87

that image scoring is not an evolutionary stable strategy, subjects follow image scoring rather than standing strategy according to empirical studies (Milinski et al., 2001). Milinski et al. (2001) argued that errors in perception and limited working memory leads to the subjects adopting image scoring strategy. Comprehensive theoretical investigations on the success of social norms in indirect reciprocity game were addressed later by (Brandt and Sigmund, 2004; Ohtsuki and Iwasa, 2004, 2006; Ohtsuki et al., 2009). Ohtsuki and Iwasa (2004, 2006), similar to most recent studies, assumed that individuals are either in Good or in Bad reputation, and they are reliably informed about the reputation state of every one. They considered all the possible third order norms, that is when an observer makes decision according to the donor and recipient reputation state and the action of the donor. (For first order norms, only the actions of the donor are taken into account, for the second order norms the reputation of either the recipient or the donor and the action also contribute to determine the new reputation value of the donor.) Thus, there are 24 different action strategies and 28 possible norms. They assumed that except for some small error individuals that are well-informed about the actions, further individuals can make some errors during execution of intended actions. They found eight reputation systems among the possible 24 × 28 = 4096, which are ESS and maintain a high level of cooperation. The common nature of these so-called “leading-eight” reputation systems is that all of them are nice (maintenance of cooperation), retaliatory (detection and punishment of defection, and justification of punishment), apologetic, and forgiving (Ohtsuki and Iwasa, 2006). In a parallel work, Brandt and Sigmund (2004) studied the evolution of only 14 different reputation systems among the possible 4096, but they studied the invasion and coexistence of strategies in a group structured individual-based model. Their main conclusion is that the standing strategy is generally superior to image scoring strategy, but standing, image scoring and a judging norms (see below) are typically in stable coexistence. Recently Ohtsuki et al. (2009) extended their previous models (Ohtsuki and Iwasa, 2004, 2006) by studying the role of costly punishment in the indirect reciprocity framework. They used second order norms, but beside cooperation and defection, punishment, as a third possible action was available. They were interested again the evolutionary stable norms which maintain high level of cooperation. They assumed that there is an inherent error in assigning reputation, but all individuals have the same opinion to a given person. They found that costly punishment is more efficient than non-punishing defection towards bad individual only at a narrow range of parameters. They studied this situation as well when everyone has a private list of the reputation of the others, and found that even small interpretation error can destroy cooperation. However, if there is a communication phase among the actions, where individuals sample each others opinion about a third party, then cooperation is maintained. Most studies emphasized that ancient human populations lived in small interacting groups (e.g. Leimar and Hammerstein, 2001; Brandt and Sigmund, 2004; Pacheco et al., 2006; Chalub et al., 2006; Scheuring, 2009), thus cooperative norms and social institutions are evolved through cultural group selection (Bowles et al., 2003; Bowles and Gintis, 2003; Pacheco et al., 2006; Chalub et al., 2006). The group selection mechanism is widely debated by arguing that group (or multilevel) selection can be explained by kin selection mechanism as well (Traulsen and Nowak, 2006; Taylor and Nowak, 2007; Lehmann et al., 2007; Traulsen et al., 2008). However, the group structure of hunter-gatherer societies is obvious (Ember, 1978; Richerson et al., 2001; Soltis et al., 1995), thus the multilevel selection perspective is natural in our case. By using multilevel selection Pacheco et al. (2006) found that evolution leads to a second (despite that individuals can follow

third order norms), named “stern-judging”. Under stern-judging norm giving help to a good individual and refusing help to a bad individual lead to a good reputation, while refusing help to a good and giving help to a bad one lead to bad reputation. Stern-judging is among the leading-eight norms found by Ohtsuki and Iwasa (2004, 2006), although stern-judging is the most successful in a multilevel selection process while leading-eight norms are only stable against the invasion of rare non-cooperative strategies under individual level selection. We note here that Chalub et al. (2006) had the same conclusion by using a similar model framework. In a recent paper Scheuring (2009) extended the definition of norm by using stochastic norms in his model. Individuals following a stochastic norm consider an action to be good with probability p (and bad with probability 1 − p). He focused on the question whether cooperative norms can evolve when populations used non-cooperative norms initially. By using a similar modeling framework that was used by Pacheco et al. (2006), and assuming that mutants can differ in the probabilities defining the norm, it is found that the evolution of norms lead to a population which follow a socalled “generous stern-judging strategy” on average. According to this strategy giving help to a good individual and refusing help to a bad individual is considered to be good with a high probability, but to help a good individual is classified to be a better action than refusing a bad. Refusing help to a good individual is a bad action with a high probability under generous judging, but with a small, but definite probability it is valued to be good. This kind of generous norm system is effectively maintained by cooperation in a system where norm polymorphism and social noise is present (Scheuring, 2009). In previous models individuals are either well informed about the social status of others (Ohtsuki and Iwasa, 2004, 2006) or only some social noise can cause misinterpretations for previous actions (Nowak and Sigmund, 1998b; Pacheco et al., 2006; Scheuring, 2009; Ohtsuki et al., 2009). Since it unlikely that individuals observe every previous (or at least the most) interaction, the information transfer (gossiping) among individuals forms the opinion of the behavior of others in the population (Ohtsuki et al., 2009). However information transfer is not necessarily reliable (Hess and Hagen, 2006), what seems to be even more plausible is that giving false or negative information about others increases the relative fitness of the gossiping people (McAndrew and Milenkovic, 2002), thus being an advantageous strategy. Consequently, to understand the origin of reliable and honest social information transfer in the context of evolution of cooperative norms is a crucial problem. The central questions of this paper are: how a reliable communication system can emerge in indirect reciprocity game? Can social norm evolve to maintain high level of indirect cooperation if information is based on indirect observation? I study these problems by extending my previous model (Scheuring, 2009), by assuming that the potential donor (partly or totally) is informed by other observers, and that individuals use information transfer as an adaptive (stochastic) strategy. In this model we can study the evolution of information strategies and the social norms, and even the evolution of information network. In the following, I introduce the model and then results of numerical simulations are presented and discussed.

2. The Model 2.1. Indirect Reciprocity Game, Reputation System Individuals play the indirect reciprocity game. The actor can give a help to the recipient, which decreases its fitness by c, while the fitness of the recipient increases by b, where b > c (Nowak and Sigmund, 1998b). (For convenience, we fixed c = 1.) If a selfish actor

I. Scheuring / BioSystems 101 (2010) 79–87

does not help the recipient then the fitness of the partners remains unchanged. The altruistic behavior is not reciprocated directly by the recipient, but this act is observed and valued by other individuals in the population. Thus, altruistic (selfish) individuals can be rewarded or punished indirectly by a third party if the previous actor becomes a potential recipient in the future. We studied a second order norm system, that is the classification of a potential recipient depends on its previous act and the reputation of their recipient (Ohtsuki and Iwasa, 2004; Pacheco et al., 2006; Scheuring, 2009). We use the simplest reputation system, thus an individual can be “Good” or “Bad”. Thus, four different acts can be classified: “Selfish” act to a “Bad” recipient (S → B), “Selfish” act to a “Good” recipient (S → G), “Altruistic” help to “Bad” individual (A → B) and “Altruistic” act to “Good” recipient (A → G). (Selfish act frequently denoted as “Defective” and Altruistic one as “Cooperative” in the literature.) 2.2. Population Structure Following the general view about the structure of ancient human population, we assume that individuals are distributed into small interacting groups, thus selection works within and among groups as well (Bowles et al., 2003; Bowles and Gintis, 2003; Pacheco et al., 2006). For simplicity, each group contains N individuals and there are M groups in our model. In addition to the probabilistic norms and actions defined below, there is another generalization in our model. While most previous models assumed that all individuals follow exactly the same social norm within a group (Nowak and Sigmund, 1998a,b; Leimar and Hammerstein, 2001; Ohtsuki and Iwasa, 2004, 2006; Pacheco et al., 2006), it is not required here (but see Brandt and Sigmund, 2004; Scheuring, 2009).

81

depending on random events, in the cooperative state either “Good” or “Bad individuals are supported with high probabilities (and the opposite with low probabilities). For convenience we denote those individuals to be “Good”, which are supported with high probability (i,j) in the evolved cooperative state (qG ≈ 1). 2.4. Information Transfer Similarly  to the stochastic norm  system, another series of probabilities

(i,j)

(i,j)

(i,j)

(i,j)

rS→B , rS→G , rA→B , rA→G

is assigned to individual i in

(i,j) rX→Y

describes the probability that an act X to group j, where individual in reputation Y is communicated to be “Good” by individual i. Naturally, information transfer is inherently noisy, thus (i,j) εr < rY < 1 − εr . Thus an individual i in group j is characterized by three vectors p(i,j) , q(i,j) , r(i,j) . We assume that the previous act of the potential receiver is observed by the potential donor with probability , otherways (with probability 1 − ) she collects the information from k randomly selected (except the donor and recipient) individuals in the population. Here we assume that the selected individuals have really observed the previous act of the potential receiver, and give unambiguous (but not sufficiently honest) information to the potential donor. That is observers inform others an individual to be either G or B with probabilities determined by their information strategy r. In the case of direct observation the donor decides to consider the recipient to be G or B according to her stochastic norm system described in the previous subsection. In the case of indirect observation the donor accepts the opinion of the majority of the observers. For example if more than half of the individuals say G for the recipient, then donor considered it to be G. (If half of the observers say G and other half say B then observer chooses randomly between G and B.)

2.3. Stochastic Norm System 2.5. The Evolutionary Mechanism We assume that the norm followed by an individual classifies acts as to be “Good” or “Bad” in a probabilistic manner. Thus, the norm of an j is determined by four prob individual i in group  abilities:

(i,j)

(i,j)

(i,j)

(i,j)

(i,j)

pS→B , pS→G , pA→B , pA→G , where pX→Y describes the

probability that an act X to individual in reputation Y is considered to be “Good” by individual i (and considered to be “Bad” with (i,j) probability 1 − pX→Y ). We assume that not only the norm is on probabilistic basis, but the actions as well. Thus, if a potential actor i using her personal norm valued a recipient to be “Good” according to the previous action of the recipient, then actor support the (i,j) recipient by the altruistic act with probability qG . If the recipient (i,j)

is observed to be “Bad”, then it is supported with probability qB . Therefore by six proba behavior of an individual i is characterized  bilities:

(i,j)

(i,j)

(i,j)

(i,j)

(i,j)

(i,j)

pS→B , pS→G , pA→B , pA→G , qG , qB p(i,j) , q(i,j)

, which are denoted

in the sequel. by the pair of vectors We assume that the interpretation of an action cannot be (i,j) perfect, thus εp < pX→Y < 1 − εp , where εp  1 is the interpretation error. Similarly, there are no purely deterministic strategies (i,j) because of execution mistakes, thus εq < qY < 1 − εq (Leimar and Hammerstein, 2001; Pacheco et al., 2006). For simplicity, we assume that εp = εq = ε in this paper. Naturally one might assume that initially a “Good” recipient is supported with a high (fixed) probability and a “Bad” one with a low (fixed) probability, but then some initial correlation between reputations and actions is built into the model. We wanted to define a completely neutral initial state without any correlation between reputation and action at the starting point of the simulations. Consequently, the meaning of “Good” and “Bad” is arbitrary, and,

Actors and recipients are selected randomly within the groups, such that every individual is once in the role of the actor and once in the role of the recipient in each round. The sums of the payoffs are considered as fitnesses of individuals. After rN rounds of interactions, there is a selection mutation step among individuals living in the same group. Pairs of randomly selected individuals are compared, and individual i imitates individual j with a probability given by =

1 , 1 + e−ˇ(wi −wk )

(1)

whereas the inverse process occurs with probability (1 − ). Here wi and wk refer to the fitness of individual i and k, respectively, ˇ determines the strength of selection (Szabó and Töke, 1998). If ˇ → ∞ then  → 1 if wi > wk , that is the less fit individual surely adopts the norm system of the fitter individual, whereas in the limit of ˇ → 0 then  → 1/2, thus there is no selection (neutral drift). Parameter ˇ is fixed ˇ = 10 in the simulations. The sensitivity of the results on this parameter is studied earlier (Scheuring, 2009). Imitation is not sufficiently perfect; if individual k adopts the norm system of individual i then the adopted norm is p(k,j) = p(i,j) + p e, where the elements of vector e are random variables with standard normal distribution, and p ∈ [0 1]. Similarly, the adopted action q(k,j) = q(i,j) + q e and the adopted communication strategy r(k,j) = r(i,j) + r e. Knowing that human individuals are norm-followers, we might assume that p , q and r  1 generally. To study the resistance of a norm against any actions, we (h) assume that q = q is close to one occasionally. To decrease the (h)

number of parameters in the model, we fixed q

= 0.8, which is

82

I. Scheuring / BioSystems 101 (2010) 79–87

Table 1 Parameter definitions.

3. Results

Notation

Description

M N K b c  r g h ε ˇ  k s

Number of groups Number of individuals within a group Number of non-assimilating individuals Benefit of cooperation Cost of cooperation Variance of nonperfect imitation rN games between two selection steps gN pairs of individuals are compared within groups hM pairs of groups are compared Interpretation, execution and information transfer error Strength of the selection Probability of direct observation Number of indirect observers Probability that number of indirect observers changes

set with probability 0.1 in the simulations, and otherwise p = q = r =   1. gN pairs of individuals are compared according to Eq. (1) in one selection mutation step, where g is a positive integer set to 10 in the simulations. After this “within group” selection step, there is selection among groups. We calculated the average fitness of individuals within groups, and compared randomly selected pairs of groups according to their average fitnesses. The fitter group replaces the less fit one by using the update rule (1), except that now  is determined by the average fitnesses of the groups. The loser group disappears, and a new group is formed with individuals following the norms of individuals of the winner group. Alternatively we might interpret it that looser are assimilated into the winner group and the winner group forms two new groups with identical size. We assume that assimilation is not complete in the beaten group, that is K individuals keep their norm system in the next generation in the newly formed group. hM randomly selected pair of groups are compared in this selection step, (that is hM must be an integer). Generally h = 1 in the simulations, but the sensitivity of the results of this assumption was studied earlier (Scheuring, 2009). After this group selection phase, a new cycle is started with indirect reciprocity games among the individuals within groups.

3.1. Fixed Social Parameters In this subsection the parameters (j) and k(j) are constant, and we assume that εp = εq = εr = 0.01. We are interested in the emergence of a cooperative norm from a non-cooperative one, thus initially all individuals are almost perfectly selfish, that is p(i,j) = 0 + 0.01e, q(i,j) = 0 + 0.01e, and they give almost meaningless information to each other r(i,j) = 0 + 0.01e for every i = 1, 2, . . . , NM. Further, initially every individual has bad (B) social status. (We found that other initial conditions lead to the same result.) First, we study the case when there is only one group (M = 1), thus selection is in action at the individual level. We made a comprehensive simulation by varying parameters, and we did not find any combination of parameters for which cooperation stably evolved within 15,000 generations. We experienced that cooperative norm can emerge only temporally and only if b is extremely high both in the direct and indirect observation case. To reveal the correlation between the characteristics of norms and the average we the average norm and action  (i,j) fitness  calculated as p¯ (j) = p , q¯ (j) = q(i,j) for every group at every time step. i i As a consequence of selection by imitation dynamics, we found that most individuals differ from each other moderately at every time step within a group, so the average norm informatively describes the norm system followed by the local population. With the help of the average norm, we can calculate the average probabilities of altruistic help of individuals being in different social status within a group as (j)

(j)

We have two parameters which determine the social connections within a group: parameter  describes the probability of direct observation and parameter k determines the number of observers transmitting information to the potential donor. In the first part of the following section we keep constant these parameters within all groups, but in the second part we assume that these parameters are inner characteristics of groups (that is  = (j), k = k(j), where j = 1, . . . , M are the indexes of the groups), and these group characters are culturally inherited and evolved. Thus at every case when a new group emerges it inherits (j), k(j) from their mother group, but occasionally mutation can occur thus (l) = (j) + e, where e is a random variable with standard normal distribution, and  means as before. Naturally (j) ∈ [0 1] after mutation. Since k(j)-s are positive integers their mutation process is different: k(j) changes to k(l) = k(j) ± 1 with probability s (s = 0.066 within the simulation). Naturally k(j) could not be smaller than 1 by definition, and could not be higher than the number of observable individuals minus the donor and the receiver (N − 2). In the simulations we determine an upper limit kmax for k(j), which can be meaningfully smaller than N − 2. It is assumed that there is no trade-off between these social parameters, although both positive and negative trade-off can be reasonable. Notations and the definition of the parameters are summarized in Table 1.

(j)

(j)

(2) (j)

if potential receiver are classified by direct observation. PX→Y is the average probability of helping an individual in group j that acts X(= A, S) on individual Y (= G, B) in the previous round. In the cases when receivers are classified by indirect observation, the average probability that an X → Y action is considered to be good is k(j) 

(j)

2.6. Coevolution of Social Parameters

(j)

PX→Y = p¯ X→Y q¯ G + (1 − p¯ X→Y )q¯ B ,

˘X→Y =



i=[k(j)/2]+1

k(j) i

 (j)

i

(j)

(¯rX→Y ) (1 − r¯ X→Y )

k(j)−i

,

(3)

(j)

where [.] is the integer function, (¯rX→Y ) is the average probability (j)

of communicating X → Y as good in group j, thus X→Y sums the weighted probabilities in that cases when more than half of the observers consider X → Y to be good in group j. Analogously to Eq. (2), the average probability of helping X → Y action in group j is (j)

(j)

(j)

(j)

(j)

QX→Y = X→Y q¯ G + (1 − X→Y )q¯ B .

(4)

Consequently, the average probability of helping X → Y in the whole population where direct and indirect observations are possible is calculated as RX→Y =

M  

(j)

(j)

(j)PX→Y + (1 − (j))QX→Y



.

(5)

j=1

In the case of purely direct observation we have seen already that the situation can be qualitatively very different if individuals are arranged into groups and selection acts among the groups as well. Then evolution typically leads to high fitness for individuals in every group because the emergence of cooperative norms (Scheuring, 2009). Now we study the evolution of norm and communication system in a group structured population, and are interested in how

I. Scheuring / BioSystems 101 (2010) 79–87

83

Fig. 1. (color online) Evolution of high fitness and the cooperative norm at different ratio of direct and indirect observation. Average of 20 independent simulations is plotted. (a) Average fitness in function of generation time at different frequency of direct observation level () is 1 (red), 0.5 (green) and 0 (blue) and there are five indirect observers (k = 5). (b) The average probabilities of altruistic help to individuals making selfish action for bad (S → B) and for a good (S → G) individual in the previous round at different values of . The color code is as before. (c) The average probabilities of altruistic help to individuals making altruistic action for bad (A → B) and for a good (A → G) individual in the previous round at different values of . The color code is as before. The parameters of the simulations are N = 64, M = 64, b = 3, K = 5,  = 0.01, r = 20. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

their evolution depends on the level of direct observation. In Fig. 1 the time evolution of norms and average fitnesses are depicted at different levels of direct observation. It can be seen that at a parameter set where high level of average fitness and cooperative norm evolves at direct observation ( = 1) there qualitatively the same does occur if observation is partly ( = 0.5) or exclusively ( = 0) indirect. What is more the higher the level of indirect observation the higher the evolved average fitness (Fig. 1a). This increased evolved fitness at indirect observation is the consequence of higher level of altruistic help to S → B and A → G actions (Fig. 1b), and similarly low level of altruism to S → G action (Fig. 1c). Because of low level of observation and interpretation errors (εs), the frequency of A → B actions are very low in the cooperative state, so practically no selection for supporting or not this behavior (Scheuring, 2009). This is the reason why RA→B is about 0.5 on average, while high variance of it indicates random fluctuations (not shown, but see Scheuring, 2009). Since the presence of indirect observers increase the level of cooperation and thus the fitness too, we studied this effect systematically by changing the number of indirect observers (Fig. 2). More observers lead to higher level of cooperation and higher average fitness (Fig. 2 inset). However, the average time needed to the transition from non-cooperative state to the cooperative one increased with the number of indirect observers and fitness increases only slightly with the number of observers (Fig. 2). Previously I introduced a simple classification for every strategy and action to reveal the evolution of norms and strategies leading to cooperative state (Scheuring, 2009). Here I use again the same classification method but extend it to the observation strategies. Since the effect of a communication strategy depends on the number of observers give the information to the donor and the strategy of the other observers within the group, we could assign a classification

to the communication strategy only on group level. The average probability that X → Y action will be altruistically help in group j because of information collected from the observers is determined (j) (j) by QX→Y (see Eq. 4). We made the following classification for QX→Y • If 0 ≤ Q (j) < 1/3, then communication leads to a “defective” X→Y strategy and is denoted by D. • If 1/3 ≤ Q (j) < 2/3, then the probability of altruistic and selfish X→Y act is roughly the same, so this communication strategy causes a “random” strategy, and is denoted by R.

Fig. 2. Average fitness in function of generation number at different number of indirect observers. Average of 20 independent simulations with  = 0, the other parameters are the same as in Fig. 1. In the inset the time average and its variance of the last 500 values (t = 4500 − 5000) are plotted.

84

I. Scheuring / BioSystems 101 (2010) 79–87

Fig. 3. (color online) The evolution of strategies using a “three-state” classification. (a) The percentage of dominant strategies in the non-cooperative state in function of the generation time. (b) The percentage of dominant strategies and their sum in the cooperative state. The parameters of the simulation are the same as in Fig. 1,  = 0, k = 5. The values are the average of ten independent simulations.

• If 2/3 ≤ Q (j) ≤ 1, then communication triggers altruistic help, X→Y thus is considered to be “cooperative” and is denoted by C.

(j)

(j)

(j)

(j)

(j)

We order QX→Y into a vector as (QS→B , QS→G , QA→B , QA→G ), and the classified probabilities are ordered in a similar manner. For example, strategies based on a “stern-judging like” norm are denoted by (C,D,D,C), while strategies based on “image scoring like” norm are denoted by (D,D,C,C). (i,j) (i,j) Similarly, qG and qB actions were classified as D, R or as C type. Thus we can have 9 different types of action pairs from the always defecting (D,D) type to the always cooperating (C,C) type. Using this three-state classification, we followed the number of groups in the different strategy and number of individuals in action classes along evolution. It is observed that the vast majority of groups, that is 80–90% of them follow only at most six different strategies among the possible 81. Because of the inherent stochasticity in the system, slightly different evolutionary pathways can be observed in different numerical experiments even at the same parameter set. By plotting the average strategy evolution of ten independent simulations, we show the typical evolutionary pathway from unconditional defection to the conditional cooperation. As Fig. 3 demonstrates there is a fast exchange of these dominant strategies when the cooperative state emerges in the system. The presence of ALLD-like strategy ((D,D,D,D)) decreases abruptly after the emergence of some strategies which catalyze the evolution of other strategies (Fig. 3a). This series of “strategy catalyzation” starts with the temporal increases of (R,D,D,D) strategy, which is followed by the (R,D,D,R) and (R,D,D,C) strategies. And the sharp transition to the cooperative state begin with the invasion of (C,D,D,C) strategy (Fig. 3a). The cooperative state is dominated by the (C,D,*,C) strategy, where (*) is a wild card, implying that all possible strategies (D,R,C) can be included here (Fig. 3b). Since initially all individuals follow ALLD-like strategies, it is not surprising that their ratio is high for a while, but interestingly about 5% of the populations remain stably ALLD-like in the cooperative state as well. The unconditionally cooperative strategies (ALLC-like or (C,C,C,C)) emerge soon after some evolutionary steps, and remain present in the population in about 5%, almost independently of the total level of cooperation in the population (not shown here, but see Scheuring (2009)). It was found that five types of actions are present at least temporarily in non-negligible fraction (more than 5%), namely the (D,D), (C,C), (C,D), (C,R) and (R,D) actions (Fig. 4). It can be seen from Fig. 4 that increase of (C,D)-s frequency are strongly correlated with the decrease of the (D,D)-s frequency. However, the invasion of (C,D) is preceded by the temporal invasion of less selective cooper-

ative action, (R,D). Beside the dominance of (C,D) in the cooperative state the defective (D,D), (R,D) and the cooperative (C,C) and (C,R) strategies are present in a non-negligible fraction. We note here that in the case of only direct observation ( = 1) we observe a very similar picture for the action evolution, except that the transition to the cooperative state is generally slower. Comparing Fig. 3a to Fig. 4 it is clear that the frequency of ALLDlike strategies is correlated with the frequency of (D,D) action, while the temporal increase of (R,D,D,*) strategies are in correlation with the frequency of (R,D) action. Similarly, by comparing Fig. 3b to Fig. 4 we can see that the emergence of cooperative judging strategies ((C,D,*,C)) is in correlation with the frequency change of (C,D). It follows from the result depicted in Fig. 2 that if the number of indirect observers can, it will increase, since the average fitness will increase. Clearly both to collect information and to observe others behavior have a cost, thus there is an upper limit for k(j)-s, which is determined by the social structure of population. Further, it is natural to assume that doubtfullness (or noise) is higher for information transfer by indirect observation than direct observation (εr > ε). The next section investigates these questions by studying the evolution of direct observation probability and the number of indirect observers.

Fig. 4. (color online) The evolution of actions using a “three-state” classification. The percentage of (D,D) (red online) and (C,D) (blue online) actions are dominant before and after the transition, respectively if there is only indirect observation ( = 0, k = 5). Actions (R,D) (green online) (C,C) (purple online) (C,R) (light blue online) are the subdominant strategies. The results are the average of ten independent numerical experiments. The other parameters of the simulation are the same as in Fig. 1. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

I. Scheuring / BioSystems 101 (2010) 79–87

85

Fig. 5. (color online) Evolution of the level of direct and indirect observations. (a) Average fitness (red), observation probability (green) and number of observers (blue) in function of generation time when εr = 0.15 and kmax = 5. (b) Average fitness (red), observation probability (green) and number of observers (blue) in function of generation time when εr = 0.3 and kmax = 5. The parameters of the simulations are as before. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

3.2. Coevolution of Social Parameters In this section we are interested in how the importance of direct and indirect observation change if mutation and selection of (j) and k(j) are allowed. We use the simple assumption that k(j) ≤ kmax < N − 2, that is, there is an upper limit for the number of opinion transfer by indirect observations. The evolution of these quantities are studied at different εr , while εp = εq are kept constant. In Fig. 5 the time evolution of average fitness, average probability of direct observation and average number of indirect observers are shown at two characteristically different εr -s. Initially (j) = 1 in both cases and k(j) = 1, that is only direct observation is present. It can be seen that if information transfer error is low then the number of observers evolves towards the possible maximum (kmax ), and probability of direct observation will be close to zero (Fig. 5a). However, if information transfer error is high then the probability of direct observation evolves close to one, while number of indirect observers fluctuates randomly between 1 and kmax (Fig. 5b). Interestingly, the suboptimal transient cooperative state evolves by keeping direct observation level high at the low information transfer error case, while this transient emerges with the help of indirect observations if information transfer error is high (see Fig. 5a and b respectively within generation time 0 and 6000). Then we studied the characteristic difference of social parameters’ evolution more systematically (Fig. 6). It can be seen that increasing information transfer error causes a transition in the evolution of social parameters. If εr becomes roughly smaller than 0.2 then it is worthwhile to base decision

on the collected information from the indirect observers, if εr is roughly greater than 0.3, then it is better to enforce the level of direct observation and does not care about the indirect ones. In the transition zone (εr is roughly between 0.2 and 0.3) the evolution sticks either to the high or the low probability of direct observation state for a very long time(about 104 generations), but as εr increases it fixes more and more in the high probability of direct observation state. High variances in this region indicate the presence of this alternative end-states within the time of simulations. I note here, that qualitatively the same result was detected if there are lower and upper level of constraints for (j)-s (0 < 1 < (j) < 2 < 1). Not surprisingly, increasing the information transfer error decreases the average fitness (Fig. 7a) just because of increased misinterpretation of actions. This increased error are in correlation with decreased probability of helping punishing (S → B) and rewarding (A → G) behavior (Fig. 7b), however the helping level of nasty (S → G) and paradoxical (A → B) behavior is practically independent of the level of εr (Fig. 7c). 4. Discussion We considered groups of individuals following a second order stochastic norm for evaluating acts of indirect altruism. Individuals are in competition with their group fellows, but there is competition among the groups as well. Further, the preceding actions of group-members (partly or exclusively) are known from the information transferred by other individuals in the group. The transferred information which is under the control of another stochastic norm, is not sufficiently reliable, even observers can

Fig. 6. Average number of indirect observers and average probability of direct observation at different information transfer errors. (Solid lines are only for visualization.) All values are averaged from 10 independent numerical experiments. The time averages and standard errors are computed from the last 500 values for simulations being 15,000 generations long. The parameters of the simulations are as before.

86

I. Scheuring / BioSystems 101 (2010) 79–87

Fig. 7. Evolved average fitness and average strategies at different information transfer errors. The same numerical experiment as in Fig. 6. The average values and standard deviations are computed as before. (a) Average relative fitness in function of information transfer error. Line is only for visualizing the trend. (b) The average probabilities of altruistic help to individuals making selfish action for bad (S → B) and altruistic for a good (A → G) individual in the previous round in function of information transfer error. (c) The average probabilities of altruistic help to individuals making altruistic action for bad (A → B) and selfish for a good (S → G) individual in the previous round at different values of εr .

cheat by transferring deliberately false or hostile information from others. Individuals by collecting information from the observers follow the majorities’ opinion to classify the behavior of the potential receiver. We have shown that a highly cooperative state evolves typically even if individuals are informed exclusively on the transferred information (e.g. indirect observation). In the evolved cooperative state the followed average norm is a generous judging strategy (Scheuring, 2009), and the information transfer will be on average honest and reliable (Fig. 1b, c). Just before the transition to the cooperative state, a series of strategies change each other. These strategies are willing to help more and more those individuals which punished defectors or were altruist in the previous round (Fig. 3a), that is, the more and more judging like strategies evolve to catalyze the invasion of a generous judging strategy. Interestingly if there is only direct observation then generous judging strategy state is attained by a different trajectory. First an image scoringlike strategy and a then standing-like strategy evolves before the emergence of generous judging strategy (Scheuring, 2009). What is more, it is advantageous to base the decision as much as possible on the information collected from the indirect observers even if the information transfer error is one order of magnitude higher than the direct observation error (Fig. 5a and Fig. 6). It is optimal to increase the number of individuals from whom the information is collected as high as possible if information transfer error is below a critical level, and to increase the level of direct observation as high as possible above this critical error level (Fig. 6). That is even a very noisy, unreliable information transfer can increase the level of cooperation if there are enough observers in the population. The explanation of this result is based on how the potential receiver is estimated from the collected information. Since the probability of considering an action X → Y to be good is determined by Eq. (3), the expected probability to consider a good action to be

good and a bad to be bad is higher even if the information transfer error is high comparing to the direct observation error. For example if εr = 0.25 then the maximal probability of considering an action to be good rmax = 1 − εr = 0.75. Assume that five observers give the information thus the good individual estimated to be good with X→Y ≈ 0.9, which accuracy can be attained by direct observation if ε = 0.1. It is generally true that X→Y > rmax if rmax ∈ (0.5 1), and at a fixed r value X→Y increases as the number of indirect observers increases. Thus the accuracy of decision can effectively be increased by following the opinion of indirect observers. The noise reducing effect of indirect observation and information transfer in indirect reciprocity game was measured experimentally as well (Sommefeld et al., 2008). The not well defined state around εr ≈ 0.25 in Fig. 6 is clearly the consequence of small fitness difference between the “pure direct observation” and “pure indirect observation” states, so the transition would be sharper if simulations have taken for longer time. Naturally as the information transfer error increase the maximal level of fitness decreases (Fig. 7a) despite that fact that evolution of social parameters can optimize the observation strategy. This fitness decreasing is correlated mainly with the decreased averaged probability to consider S → B action and in a less pronounced manner the A → G action to be good (Fig. 7b), but not with the awarding of S → G and A → B actions (Fig 7c). Thus the increased noise in the information processing mainly decreases the rewarding of punishing and rewarding behavior. Our aim was to create a relatively simple strategic model, thus it can be extended or modified at several details. In the following I collect some points which can arise if modifications are made: • It is likely that (j) and k(j) are in trade-off in real groups. If this trade-off is positive (more direct observation leads to more communicated indirect observations), then both (j) and k(j) will

I. Scheuring / BioSystems 101 (2010) 79–87

evolve toward the maximal value. If the trade-off is negative, then we will get qualitatively the same result as in our trade-off free model. • Gossip, or information transfer from others are mainly about the norm deviance (Hess and Hagen, 2006). Naturally this behavior increases the fitness of the communicator indirectly, further it strengthens the norm within the population. However it is sufficient for a reliable information transfer too, simply because individuals who are not under gossip are definitely “good” individuals according to the group’s norm (“no news, good news”). We would use this type of communication system, but the main conclusions are hardly sensitive to this detail. • All transferred information is taken into account with the same weight in our model, which is not the case in real world: stories of hostile, unreliable gossipers are less important than of reliable ones. Thus we extend the model by assuming that individuals whom opinions belong to the majority from a point of view of a potential donor will be considered with a higher weight by this donor in the next, and individuals belonging to the minority will be weighted less in the following. According to our preliminary studies, this modification causes no qualitative difference in the norm evolution, except that transition to the cooperative state is faster than the original model was. Our model is based on the assumption that individuals follow inherently stochastic social norms. It is clear from experiments that subjects do not give deterministic “yes” or “no” decisions to a given situation in indirect reciprocity or in public goods games (Wedekind and Milinski, 2000; Milinski et al., 2001; Fehr and Fischbacher, 2003; Bolton et al., 2005). In Milinski et al.’s experiment (Milinski et al., 2001), subjects play the indirect reciprocity game in groups. One of the subjects in each group is secretly instructed to be always selfish. These players are punished almost with 100% after some rounds by doing selfish act, but the few altruistic donors of these selfish individuals are donated consistently with about 70% (PA→B ≈ 0.7). This latter value is hardly the consequence of misinterpretation, it is rather the sign of norm stochasticity and/or norm polymorphism. Bolton et al.’s work (2005) serves an even stronger background for our assumption. They focused on the effect of information limitations on the amount of cooperation in a set of indirect reciprocity games, and they never found that any PX→Y probabilities are close to zero or one implying deterministic norm. The other group of experiments (Sommefeld et al., 2007, 2008) focused on the role of gossip in maintaining indirect reciprocity. They found that information transfer is on average honest, subjects follow the majority’s opinion, and more independent observers increase the accuracy in decision and the level of cooperation. All of these results are in accordance with our results. Acknowledgments This work was founded by OTKA T049692 and NN71700 (TECT).

87

References Alexander, R.D., 1987. The Biology of Moral Systems. Aldine de Gruyter, New York. Bolton, G.E., Katok, E., Ockenfels, A., 2005. Cooperation among strangers with limited information about reputation. J. Publ. Econ. 89, 1457–1468. Bowles, S., Choi, J.-K., Hopfensitz, A., 2003. The co-evolution of individual behaviors and social institutions. J. Theor. Biol. 223, 135–147. Bowles, S., Gintis, H., 2003. Origins of human cooperation. In: Hammerstein, P. (Ed.), Genetic and Cultural Evolution of Cooperation. The MIT Press, Cambridge, England. Brandt, H., Sigmund, K., 2004. The logic of reprobation: assessment and action rules for indirect reciprocation. J. Theor. Biol. 231, 475–486. Chalub, F.A.C.C., Santos, F.C., Pacheco, J.M., 2006. The evolution of norms. J. Theor. Biol. 241, 233–240. Ember, C., 1978. Myths about hunter-gatherers. Ethnology 17, 439–449. Fehr, E., Fischbacher, U., 2003. The nature of human altruism. Nature 425, 785– 791. Hess, N.H., Hagen, E.H., 2006. Psychological adaptations for assessing gossip veracity. Hum. Nat. 17, 337–354. Lehmann, L., Keller, L., West, S., Roze, D., 2007. Group selection and kin selection: two concepts but one process. Proc. Natl. Acad. Sci. U.S.A. 104, 6736–6739. Leimar, O., Hammerstein, P., 2001. Evolution of cooperation through indirect reciprocity. Proc. R. Soc. B 268, 745–753. McAndrew, F.T., Milenkovic, M.A., 2002. Of tabloids and family secrets: the evolutionary psychology of gossip. J. Appl. Soc. Psychol. 32, 1064–1082. Milinski, M., Semmann, D., Bakker, T.C., Krambeck, H.J., 2001. Cooperation through indirect reciprocity: image scoring or standing strategy? Proc. R. Soc. B 268, 2495–2501. Nowak, M., Sigmund, K., 1998a. The dynamics of indirect reciprocity. J. Theor. Biol. 194, 561–574. Nowak, M., Sigmund, K., 1998b. Evolution of indirect reciprocity by image scoring. Nature 393, 573–577. Ohtsuki, H., Iwasa, Y., 2004. How should we define goodness?-reputation dynamics in indirect reciprocity. J. Theor. Biol. 231, 107–120. Ohtsuki, H., Iwasa, Y., 2006. The leading eight: social norms that can maintain cooperation by indirect reciprocity. J. Theor. Biol. 239, 435–444. Ohtsuki, H., Iwasa, Y., Nowak, M., 2009. Indirect reciprocity provides only a narrow margin of efficiency for costly punishment. Nature 457, 79–82. Pacheco, J.M., Santos, F.C., Chalub, F.A.C.C., 2006. Stern-judging: a simple, successful norm which promotes cooperation under indirect reciprocity. PLOS Comp. Biol. 2 (e178), 1634–1638. Panchanathan, K., Boyd, R., 2003. A tale of two defectors: the importance of standing for evolution of indirect reciprocity. J. Theor. Biol. 224, 115–126. Panchanathan, K., Boyd, R., 2004. Indirect reciprocity can stabilize cooperation without the second order free-rider problem. Nature 432, 499–502. Richerson, P., Robert, B., Bettinger, R.L., 2001. Was agriculture impossible during the pleistocene but mandatory during the holocene? a climate change hypothesis? Am. Antiquity 66, 387–411. Scheuring, I., 2009. Evolution of generous cooperative norms by cultural group selection. J. Theor. Biol. 257, 397–407. Soltis, J., Boyd, R., Richerson, P.J., 1995. Can group-functional behaviours evolve by cultural group selection? an empirical test. Curr. Anthropol. 36, 473–494. Sommefeld, R.D., Krambeck, H.J., Milinski, M., 2008. Multiple gossip statements and their effect on reputation and trustworthiness. Proc. R. Soc. B 275, 2529– 2536. Sommefeld, R.D., Krambeck, H.J., Semmann, D., Milinski, M., 2007. Gossip as an alternative for direct observation in games of indirect reciprocity. Proc. Natl. Acad. Sci. 104, 17435–17440. Szabó, G., Töke, C., 1998. Evolutionary prisoner’s dilemma game on a square lattice. Phys. Rev. E 58, 69–73. Taylor, C., Nowak, M., 2007. Transforming the dilemma. Evolution 61, 2281– 2292. Traulsen, A., Nowak, M., 2006. Evolution of cooperation by multilevel selection. Proc. Natl. Acad. Sci. U.S.A. 103, 10952–10955. Traulsen, A., Shoresh, N., Nowak, M., 2008. Analytical results for individual and group selection of any intensity. Bull. Math. Biol. 70, 1410–1424. Trivers, R., 1985. Social Evolution. Benjamin, Cummings, Menlo Park, CA. Wedekind, C., Milinski, M., 2000. Cooperation through image scoring in humans. Science 288, 850–852.