Mutation Rates and Equilibrium Selection under Stochastic Evolutionary Dynamics Ryoji Sawa∗ Department of Economics, University of Wisconsin-Madison

June 25, 2011

Abstract We examine an evolutionary model in which the mutation rate varies with the strategy. Bergin and Lipman (1996) show that equilibrium selection using stochastic evolutionary processes depends on the specification of mutation rates. We offer a characterization of how mutation rates determine the selection of Nash equilibria in 2 × 2 symmetric coordination games for single and double limits of the small mutation rate and the large population size. We prove that the restrictions on mutation rates which ensure that the risk-dominated equilibrium is selected are the same for both orders of limits. Keywords: Evolutionary game theory; equilibrium selection; large populations; large deviations. JEL Classification Numbers: C72, C73.

1

Introduction

We study an evolutionary model with state-dependent mutation rates. Kandori et al (1993) (KMR henceforth) and Young (1993) showed that evolutionary processes with a common mutation rate always select risk-dominant equilibria in 2 × 2 symmetric normal form games. This unique prediction result was criticized by Bergin and Lipman (1996), who showed that the evolutionary process can select any equilibrium if we allow the mutation rates to be state-dependent. However, these authors do not attempt to characterize the relationship between mutation rates and equilibrium selection. We obtain a precise characterization of mutation rates under which a risk-dominated equilibrium will be selected in 2 × 2 symmetric normal form games. While Bergin and Lipman (1996) allows an arbitrary dependence of mutation rates on the population state to get their general result, we specify a mutation rate for each strategy to get an explicit relationship between mutation rates and equilibrium selection. ∗ Department of Economics, University of Wisconsin-Madison 1180 Observatory Drive, Madison, WI 53706-1393, United States. Tel. +1 608.262.0200. email: [email protected]

1

To illustrate our analysis concretely, suppose that n agents play the coordination game displayed in Table 1. A denotes ’Arrive on time’, while B denotes ’Be late’. If both choose A, then they meet on time and Table 1: Meeting Game

1

A B

2 A 5, 5 0, 0

B 0, 0 4, 4

each earns 5. If each chooses a different strategy, then they fail to meet and each earns 0. If both choose B, they meet late and are less productive compared to (A, A). A is both risk and payoff dominant. Let εs > 0 denote the mutation rate at which an agent takes strategy s ∈ {A, B}. If εA = εB , KMR shows that the all-A state is the unique long-run equilibrium. But, personal experience tells us that people are more likely to be unintentionally late than unintentionally early. Thus, strategy-dependent mutation rates, such as εA < εB , would be appropriate. Our theorem shows, for example, that when εB = 0.05, any εA < 0.022 guarantees that the all-B state is the unique long-run equilibrium when the population size is large enough. In next section, we first analyze the small mutation rate limits, characterizing the stochastically stable state as the strategy-dependent mutation rates approach zero. Then, we follow Binmore and Samuelson (1997) and analyze the large population limits. In contrast to Binmore and Samuelson (1997) and papers noted next, agents simultaneously update their strategies in the present model. We also consider the double limit case in which both the mutation rate and the population size limits are taken sequentially, and find that the selection results coincide for both orders of limits. This result is consistent with Fudenberg and Hojman (2009) and Sandholm (2010), which also consider the robustness of the limiting stationary distribution to the order of limits in models of noisy best response dynamics.

2

Model and Results

We consider a game in which n < ∞ agents are randomly matched to play a stage game in each period t = {1, 2, . . .}.1 We restrict our attention to 2 × 2 symmetric coordination games illustrated in Table 2. We assume that payoffs satisfy a > c, d > b, and a − c > d − b, so that (A, A) and (B, B) are symmetric Nash equilibria and strategy A is risk dominant. Let v∗ =

1 a−c > (a − c) + (d − b) 2

be the probability of playing B in the mixed strategy equilibrium of the stage game. Then v∗ is also the 1 It

is assumed that players can be matched against themselves. Similar results hold without self-matching.

2

Table 2: A 2 × 2 Coordination Game

1

A B

2 A a, a c, b

B b, c d, d

critical value such that the agent’s best response is B if the proportion of agents playing B is more than v∗ . We now introduce an evolutionary model following KMR and Bergin and Lipman (1996). The state ωt

∈ {0, 1/n, . . . , (n − 1)/n, 1} identifies the fraction of agents who plan to play B in period t.2 Let eA and

eB denote the unanimous states eA = 0 and eB = 1, which correspond to the two pure strategy equilibria. We consider the best response dynamic with mutations. For s ∈ {A, B}, let εs > 0 denote the strategydependent mutation rate.3 Play proceeds as follows. To start round t, each agent who plans to play s mutates to s0 6= s with probability εs0 > 0. Thus, each agent plays their planned strategy s with probability 1 − εs0 , and plays s0 with probability εs0 . After this, agents simultaneously update, choosing the best response against the post-mutation distribution of strategies. We assume that an agent switches her strategy if and only if her (planned) strategy is not a best response.4 For example, an agent playing A will switch if and only if the post-mutation fraction of strategy B is strictly larger than v∗ . Note that at the end of each period, all agents play the same strategy. Next, let (ω, ω 0 ) denote a transition of states from ω to ω 0 and p (ω, ω 0 ) denote the transition probability of (ω, ω 0 ). Let bxc denote the largest integer less than or equal to x. With the best response dynamic, we can view the Markov chain as having two states, eA = 0 and eB = 1 with transition probabilities n

p (eA , eB ) =

  n ∑ j (1 − εB )n− j εBj , j= jA

(1)

n

p (eB , eA ) =

  n ∑ j (1 − εA )n− j εAj , j= jB

(2)

where jA = bn (1 − v∗ )c + 1 and jB = bnv∗ c + 1. Since εs > 0, p (ω, ω 0 ) is positive valued for any ω, ω 0 ∈ ∞ is irreducible and aperiodic, and so admits unique stationary distribution {eA , eB }. Thus, the process {ω t }t=1

πε,n . Let πε,n (ω) denote the positive mass that the stationary distribution places on state ω. The limiting 2 The definitions of the state space and the limiting stationary distribution follow Binmore et al (1995), Binmore and Samuelson (1997) and Sandholm (2010). 3 A set of strategy-dependent mutation rates is a subset of state-dependent mutation rates. Bergin and Lipman (1996) similarly defined strategy-dependent mutation rates in their example. 4 Agents are assumed to stick their planned strategy when a payoff-tie occurs. If nv∗ ∈ Z, Theorem 1 needs a slight modification for a different tie breaking rule. However, Theorem 3 and Corollaries 2 and 5 hold for any arbitrary tie breaking rule.

3

stationary distributions of πε,n are defined as follows: πn0 = lim πε,n , π 0,∞ = lim lim πε,n , πε∞ = lim πε,n , and π ∞,0 = lim lim πε,n n→∞ ε→0

ε→0

n→∞

ε→0 n→∞

(3)

where ε → 0 denotes that εA → 0 and εB → 0. The last three limits in (3) refer to weak convergence of probability measures. If we restrict mutation rates to be constant across states, i.e. εA = εB , then πn0 (eA ) = 1 as shown in KMR. More generally, the following theorem offers a precise characterization of the mutation rates leading to the selection of each equilibrium.5 Theorem 1 Suppose that the stage game is a coordination game displayed in Table 2. Let n < ∞ be the population size. Define

bnv∗ c + 1 . bn (1 − v∗ )c + 1
α ∞  Then, if a sequence of mutation rate pairs εAk , εBk k=0 satisfies εAk ≤ εBk for all k and some α > α ∗ , then
α πn0 (eB ) = 1. If the opposite inequalities hold, εAk ≥ εBk for some α < α ∗ , then πn0 (eA ) = 1. α∗ =

Proof. First, note that the stationary distribution of each state is computed as, πε,n (eA ) =

p (eB , eA ) p (eA , eB ) and πε,n (eB ) = . p (eA , eB ) + p (eB , eA ) p (eA , eB ) + p (eB , eA )

(4)

Suppose that εA = εBα . Transition probability (2) is rewritten as follows: n

p (eB , eA ) =

  n ∑ j (1 − εBα )n− j εBα j j= jB

(5)

Observe that the ratio of transition probabilities is written as " p (eB , eA ) = p (eA , eB )

εBα jB − jA

n
jB + n
jA +

n− jB

∑

j=1

n− jA

∑

j=1

# n
α n− jB − j ε α j B jB + j (1 − εB )

.

n− jA − j j n
εB jA + j (1 − εB )

This implies that p (eB , eA ) /p (eA , eB ) → 0 as εA , εB → 0 if bnv∗ c + 1 < α (bn (1 − v∗ )c + 1). Equation (4) implies that πn0 (eB ) = 1. We can prove the opposite case in a similar way. The following corollary extends Theorem 1 to the case in which the small mutation limits are taken first and the large population limit is taken second.
the model without self-matching, replace v∗ with v∗∗ = a − c + n−1 (d − a) / (a − c + d − b) . If v∗∗ < 1/2, eB can be the ∗ long-run equilibrium even if εA = εB (cf. Sandholm (1998)). In this case, α < 1 may hold. 5 For

4

 ∞
α Corollary 2 If a sequence of mutation rates εAk , εBk k=0 satisfies εAk ≤ εBk for all k and some α > α ∗∗ where α ∗∗ =

v∗ , 1 − v∗

then π 0,∞ (eB ) = 1. If the opposite inequalities hold, then π 0,∞ (eA ) = 1. Proof. Observe that

bnv∗ c + 1 v∗ = = α ∗∗ . ∗ n→∞ bn (1 − v )c + 1 1 − v∗

lim α ∗ = lim

n→∞

Then the result is immediate from Theorem 1. Up to this point, we have emphasized small mutation limits. Next, we ask whether a similar characterization can be obtained if we fix the mutation rates and identify the limiting distribution as n → ∞, as in Binmore and Samuelson (1997). The following theorem answers this question in the affirmative. Theorem 3 Consider the game illustrated in Table 2. Assume εA and εB are such that max{εA , εB } < 1 − v∗ . πε∞ (eB ) = 1, if (1 − v∗ ) log εA − v∗ log εB + v∗ log (1 − εA ) − (1 − v∗ ) log (1 − εB ) < 0,

(6)

and πε∞ (eA ) = 1 if the opposite inequality holds. Note that since v∗ > 21 , condition (6) implies that εA < εB . This condition makes a transition (eB , eA ) less likely than an opposite transition. As in Sandholm and Pauzner (1998), we use a large deviation theorem to provide a simple proof.

6

Proof. For the first case, it suffices to show that lim

n→∞

1 p (eA , eB ) log = δ > 0. n p (eB , eA )

Then, for any sufficiently large n, there exists δ 0 > 0 such that
p (eA , eB ) p (eA , eB ) 1 log > δ0 ⇔ > exp nδ 0 . n p (eB , eA ) p (eB , eA ) Then, according to Equation (4), it follows that πε∞ (eB ) = 1. Let Sns denote the total number of agents playing s ∈ {A, B}. Transition (eA , eB ) requires more than nv∗
agents to play B by mutations.7 Observing that p (eA , eB ) = P SnB > nv∗ , Theorem 9.3 of Billingsley (1995) 6 Sandholm and Pauzner (1998) also shows that the process can converge to e with probability one if the population is growing B over time. 7 We assume that nv∗ is not an integer. Theorem 3 still holds for nv∗ ∈ Z with a little modification of the proof.

5

tells us that p (eA , eB ) is characterized as follows:  B  Sn v∗ 1 1 − v∗ ∗ > v = −v∗ log − (1 − v∗ ) log lim log P n→∞ n n εB 1 − εB

(7)

Similarly, transition (eB , eA ) requires more than n (1 − v∗ ) agents to play A. p (eB , eA ) is computed as  A  1 Sn 1 − v∗ v∗ ∗ lim log P > 1 − v = − (1 − v∗ ) log − v∗ log . n→∞ n n εA 1 − εA

(8)

Equations (7) and (8) along with the discussion at the beginning suggest that eB is the unique long-run equilibrium if  B   A  1 Sn 1 Sn ∗ ∗ lim log P > v > lim log P > 1−v n→∞ n n→∞ n n n ⇔ (1 − v∗ ) log εA − v∗ log εB + v∗ log (1 − εA ) − (1 − v∗ ) log (1 − εB ) < 0. The proof of the opposite case is similar. Theorem 3 shows sharp equilibrium selection results as n → ∞. Fixing εB > 0, there exists εA∗ such that πε∞ (eB )

= 1 for any εA < εA∗ and πε∞ (eB ) = 0 for any εA > εA∗ . We illustrate the nature of Theorem 3 by an

example below. Example 4 (Numerical) Consider the game in Table 1. Note that, given εB = 0.05, the upper bound that satisfies condition (6) is approximately εA ≤ .023. Figure 1 shows how πε,n (eB ) converges to one as n becomes large for εA = 0.02 and εB = 0.05. Figure 2 shows πε,n (eB ) as a function of εA for εB = 0.05 and various population sizes. πε,n (eB ) approaches a step function as n approaches infinity. Figures 1 and 2 show that if condition (6) holds, then the risk dominated equilibrium has the larger mass in πε,n even if n is relatively small, e.g. πε,n (eB ) ≈ 0.995 for n = 50, εA = 0.02 and εB = 0.05.8 Our final result relates the characterization of mutation rates under small mutation limits and large population limits. Corollary 5 shows that under both orders of limits, the same condition on mutation rates determines equilibrium selection.  ∞
α Corollary 5 If a sequence of mutation rates εAk , εBk k=0 satisfies εAk ≤ εBk for all k and some α > α ∗∗ =
α v∗ / (1 − v∗ ), then π ∞,0 (eB ) = π 0,∞ (eB ) = 1. If the opposite inequality holds, i.e. εAk ≥ εBk for some α < α ∗∗ , then π ∞,0 (eA ) = π 0,∞ (eA ) = 1. 8 In Figure 1, note that π ε,n (eB ) is not monotonically increasing over n, because the number of mutating agents required to move the state to another is not proportional to n as a result of integer effects. For instance, if n = 5, 3 mutants are required for both transitions, (eA , eB ) and (eB , eA ). If n = 6, at least 4 mutants are required for (eA , eB ), while 3 mutants are required for (eB , eA ). The transition probabilities are approximately computed as

p (eA , eB ) ≈ 125 × 10−6 , −6

p (eA , eB ) ≈ 6.25 × 10

,

p (eB , eA ) ≈ 8 × 10−6

for n = 5

p (eB , eA ) ≈ 8 × 10−6

for n = 6

The difficulty of escaping from eA becomes greater when n changes from 5 to 6, but the difficulty of escaping from eB remains the same. As a consequence, πε,5 (eB ) > πε,6 (eB ) holds.

6

1

π ε,n(eB)

0.8 0.6 0.4 0.2

0

10

20

30

40

50

n

60

70

80

90

100

Figure 1: Stationary distribution πε,n (eB ) as a function of the population size n for (εA , εB ) = (0.02, 0.05) . Proof. As εA , εB → 0, the last two terms of LHS of condition (6) converge to 0. Let εA = εBα . Condition (6) is then reduced to (1 − v∗ ) log εBα − v∗ log εB < 0 ⇔ α > α ∗∗ =

v∗ . 1 − v∗

Theorem 3 implies that π ∞,0 (eB ) = 1. With Corollary 2, the result follows. Acknowledgement I am very grateful to William H. Sandholm, Valko Benedek, an associate editor and an anonymous referee for their valuable comments and suggestions.

References Bergin J, Lipman BL (1996) Evolution with state-dependent mutations. Econometrica 64:943–956 Billingsley P (1995) Probability and Measure, 3rd edn. John Wiley & Sons Binmore K, Samuelson L (1997) Muddling through: Noisy equilibrium selection. Journal of Economic Theory 74:235–265 Binmore K, Samuelson L, Vaughan R (1995) Musical chairs: Modeling noisy evolution. Games and Economic Behavior 11:1–35 Fudenberg D, Hojman D (2009) Stochastic stability in large populations Unpublished Manuscript

7

1

π ε,n (eB)

0.8 0.6 0.4 n=500

0.2

0

0.01

0.02

n=50

εεA A

0.03

n=10

0.04

0.05

Figure 2: Stationary distribution πε,n (eB ) as a function of the mutation rate εA for εB = 0.05. Kandori M, Mailath GJ, Rob R (1993) Learning, mutation, and long run equilibria in games. Econometrica 61:1003–1037 Sandholm WH (1998) Simple and clever decision rules for a model of evolution. Economics Letters 61:165– 170 Sandholm WH (2010) Orders of limits for stationary distributions, stochastic dominance, and stochastic stability. Theoretical Economics 5:1–26 Sandholm WH, Pauzner A (1998) Evolution, population growth, and history dependence. Games and Economic Behavior 22:84–120 Young HP (1993) The evolution of conventions. Econometrica 61:57–84

8