Int J Game Theory (2001) 30:221–245
2001 99 9 9
3-player repeated games with lack of information on one side Je´roˆme Renault* CEREMADE, Universite´ Paris IX Dauphine, Place du Mare´chal de Lattre de Tassigny, 75775 Paris Cedex 16, France. This work has mostly been carried out at CERMSEM, Universite´ Paris I. (e-mail:
[email protected]) Final version June 2001
Abstract. We study the existence of uniform equilibria for three-player repeated games with lack of information on one side and perfect observation. If there are only two states of nature, a completely revealing or a joint plan equilibrium always exists. This is not the case for larger spaces of states. Key words: repeated games, incomplete information, lack of information on one side, 3 players, uniform equilibrium, existence of equilibria.
1. Introduction Repeated games with incomplete information were introduced by Aumann and Maschler in the sixties (see [Au-Ma]). For two-player games with perfect observation, they showed that uniform equilibria may fail to exist if none of the players has more information than the other. However, in the opposite case of lack of information on one side, the question of the existence of equilibria has been positively answered in 1995 by Simon, Spiez˙ and Torun´czyk ([Si-Sp-To]). We are interested in the question, only posed here, of the generalization of these results for the n-person case. Which structures of information ensure the existence, for all payo¤ functions, of uniform equilibria? For example, one can in a first approach consider the case of ‘‘information in chain’’, where the players can be totally ordered from the most informed one to the least informed one. We only deal here with the apparently simple case of repeated games with lack of information on one side and perfect observation. In this model, a finite game is first chosen according to a known probability, and is announced to * I wish to thank Prof. J. Abdou for his supervision.
222
J. Renault
all the players but one, called the uninformed player. Then the selected game is repeated, and after each stage the actions just played are announced in a public way. Assuming that the repetition is infinite and players are infinitely patient, we use the notion of uniform equilibrium. We can then concentrate on long-term strategic aspects. For example, the beginning of the play can be devoted to communication without influencing payo¤s. We insist on the fact that we just look here for the existence of some (if possible simple) equilibrium. If at least four players are involved, it is easy to see that a completely revealing equilibrium (i.e. an equilibrium where the uninformed player eventually learns the true state of nature) always exists: the play can start with a revelation phase where each informed player announces, by following some code on actions, the selected game. Since there are at least three informed players, even if one of them deviates the uninformed player will be able to deduce the selected game (see Remark 2.2 for more details). This is no longer true if there are only two informed players, because if they announce two di¤erent states the uninformed player can not find out who did deviate and hence what the true state is. We thus only consider three-player repeated games with lack of information on one side, hence at the beginning of the interaction two players (fixed in advance) are informed of the selected game. We show that in the case of two states of nature, i.e. when only two finite games are a priori possible, equilibria always exist. Moreover, if no completely revealing equilibrium exists, some simple equilibrium called a joint plan equilibrium will always do. This notion of joint plan is the simplest generalization of the one introduced by Aumann, Maschler and Stearns (see [Au-Ma] again) for the two-player case. Joint plans were used by Sorin ([So]), for two states of nature, and by Simon, Spiez˙ and Torun´czyk, for an arbitrary number of states, to prove the existence of equilibria in two-player games. They represent here the situations of a partial revelation of information (i.e. the selected state of nature), made in a unique step by one of the informed players without being controlled by the other. For larger spaces of states, we show that this is no longer the case: completely revealing equilibria and joint plans may simultaneously fail to exist. We conclude with the consideration of other classes of equilibria, which might always exist for this 3-player case. 2. The model If S is a finite set, jSj will denote its cardinality and DðSÞ the set of probability distributions over S. DðSÞ will be viewed as fz ¼ ðz s Þs A S A RS: Es A S; P s s z b 0 and s A S z ¼ 1g. An element z in DðSÞ may also be denoted by z will denote the supz ¼ ðzðsÞÞs A S or by z ¼ ðzs Þs A S . For z in DðSÞ, SuppP port of z, and for x ¼ ðx s Þs A S in RS , kxk will denote s A S jxðsÞj. A three-person repeated game with lack of information on one side and perfect observation is given by the following data: – Three players, namely player 1 and 2 (the informed players) and player 3 (the uninformed player). – For each player i, a set of actions Si . – A finite non empty set K (set of states), and p0 ¼ ðp0k Þk A K a (initial) probability on K. – For each player i and state k, a payo¤ function gik from the cartesian product S1 S2 S3 (also denoted by S) to R.
3-player repeated games with lack of information on one side
223
The play of the infinitely repeated game is the following: – at stage 0, k in K is selected according to p0 and announced to the informed players. – at each stage t ¼ 1; 2; . . . , the players independently choose an action in their own set of actions. If st A S is the joint action selected, then each player i has a stage payo¤ given by gik ðst Þ. Before starting stage t þ 1, all players learn st . Note that after each stage, the informed players can deduce the stage payo¤s. This is not the case for the uninformed player, who in general may not even know his own stage payo¤. Players are assumed to have perfect recall, and the whole description of the game is public knowledge. We assume that each set of actions is finite, and, w.l.o.g., that the initial probability p0k of any state k is positive. Since a player with a single action has no influence, to really deal with 3-player games we also assume that each player has at least two actions. Denote by G y ð p0 Þ the repeated game with incomplete information just defined. Put N ¼ f1; 2; 3g (set of players) and IN ¼ f1; 2g (set of informed players). Let, for any t b 0, Ht denote the cartesian product S t of player 3’s histories up to stage t (H0 standing for a singleton). A behavior strategy for an informed player i in IN is an element si ¼ ðsik Þk A K where for each k in K, sik is a mapping from 6tb0 Ht to DðSi Þ, sik ðht Þ giving the lottery on actions to be played by player i at stage t þ 1 if k is the state of nature and ht has previously occurred. Similarly, a behavior strategy for the uninformed player is a mapping s3 from 6tb0 Ht to DðS3 Þ. Denote by Si the set of behavior strategies of player i, for each i, and by S the product S1 S2 S3 . Let Hy ¼ S Nnf0g be the set of player 3’s infinite histories. For any t b 0, we denote by Ht the s-algebra on Hy generated by the projection to Ht . Given a fixed state k, a joint (behavior) strategy s ¼ ðs1 ; s2 ; s3 Þ in S naturally induces a probability distribution over Hy , endowed with the s-algebra Hy generated by ðHt Þtb0 . This probability distribution actually only depends on s k , defined as ðs1k ; s2k ; s3 Þ, and will be denoted by Ps k . Similarly, s and p0 induce a probability distribution Pp0 ; s over the set of plays H y ¼ K Hy , endowed with the product s-algebra Hy ¼ 2 K n Hy . We can thus put (the tilde denoting random variables, s~t standing for the element in S played at stage t), for all players i, states k and stages T b 1: ! T 1X k k gi; T ðsÞ ¼ Es k g ð~ st Þ ; T t¼1 i gi; T ðsÞ ¼ Ep0 ; s
T 1X ~ g k ð~ st Þ T t¼1 i
! ¼
X
p0k gi;k T ðsÞ:
kAK
We assume that players only consider the expectation of their (stage-) average payo¤s, and use the following notion of equilibrium. As usual, Q if s ¼ ðsi Þi A N A S and i A N, s i stands for the element ðsj Þj0i of S i ¼def j0i Sj . Definition 2.1. s A S is a uniform equilibrium of G y ð p0 Þ if:
224
J. Renault
(i) For all e > 0, there exists a positive integer T0 such that s is an e-Nash equilibrium in finitely repeated games with at least T0 stages, that is: ET b T0 ; Ei A N;
gi; T ðs i ; ti Þ a gi; T ðsÞ þ e
Eti A Si ;
ðgi;k T ðsÞÞTb1
(ii) For any i in IN and k in K, converges as T goes to infinity to some gik ðsÞ, and ðg3; T ðsÞÞTb1 converges as T goes to infinity to some g3 ðsÞ. The tuple ððg1k ðsÞÞk A K ; ðg2k ðsÞÞk A K ; g3 ðsÞÞ in RK RK R is then called an equilibrium payo¤ of G y ðp0 Þ. As for two-player games (see [Ha]), there is an asymmetry concerning payo¤s, due to the fact that the informed players can maximize their payo¤s in each state. Our problem is to prove, for any such game, the existence of some (preferably simple) uniform equilibrium. Let, for any state k, G k denote the (mixed extension of the) 3-player one-shot game given by the payo¤s functions g1k , g2k and g3k . As usual, for i in N, an element of DðSi Þ will be called a mixed action of player i. Remark 2.2: n-player repeated games with lack of information on one side A more general model can be written. Assume that the set of players is N ¼ f1; 2; . . . ; ng for some n b 2 and that players 1; 2; . . . ; n 1 are informed whereas player n is not. Then all the previous definitions can be generalized in an obvious way. Recall that if n ¼ 2, the existence of uniform equilibria has been proved in [Si-Sp-To]. If n b 4, it is easy to construct a uniform equilibrium as follows. Fix, for any state k, a Nash equilibrium in mixed strategies of G k , ðxik Þi A N . Each informed player starts with revealing the selected state k (using codes on actions). This lasts a finite number of stages, say T, meanwhile the uninformed player plays arbitrarily. Since there are at least 3 informed players, the uninformed player is then able to deduce k, even in case of unilateral deviation: he just has to identify k with any state revealed by at least 2 informed players. Afterwards, every player i simply plays forever at each stage according to xik (if no state was revealed by at least 2 informed players, the uninformed player plays arbitrarily). This constitutes a uniform equilibrium because even if some player j unilaterally deviates from this strategy, at every stage t > T all other players rr will play according to ðxik Þi A Nnf jg , with k being the selected state. Back to our three-player case, we now consider a very simple example showing that the simple idea of the strategy of remark 2.2 is not su‰cient if there are only two informed players. Example 2.3. p0 ¼ ð1=2; 1=2Þ r l
l
r
u
ð1; 1; 0Þ
ð1; 1; 0Þ
u
ð0; 0; 1Þ
ð0; 0; 1Þ
d
ð1; 1; 0Þ
ð1; 1; 0Þ
d
ð0; 0; 1Þ
ð0; 0; 1Þ
L
state a
R
3-player repeated games with lack of information on one side
l
r
u
ð0; 0; 1Þ
ð0; 0; 1Þ
d
ð0; 0; 1Þ
ð0; 0; 1Þ L
l
r
u
ð1; 1; 0Þ
ð1; 1; 0Þ
d
ð1; 1; 0Þ
ð1; 1; 0Þ
state b
225
R
The interpretation of these matrices is the following. There are two states of nature a and b, and p0a ¼ p0b ¼ 1=2. Player 1 chooses the row (S1 ¼ fu; dg), player 2 chooses the column (S2 ¼ fl; rg), and player 3 the matrix (S3 ¼ fL; Rg). If, for example, b is the true state and ðu; r; LÞ is chosen at some stage, then the corresponding payo¤ is ð0; 0; 1Þ, meaning that g1b ðu; r; LÞ ¼ 0 (first coordinate), g2b ðu; r; LÞ ¼ 0 (second coordinate), and g3b ðu; r; LÞ ¼ 1 (third coordinate). Assume that s in S is such that, as in the previous remark, at the beginning of the repetition the informed players announce the true state. We claim that s can not be a uniform equilibrium and now only sketch the proof (for a formal proof it is just a consequence of proposition 4.3). Assume that the contrary holds. Then we must have g3 ðsÞ ¼ 1, since player 3 will eventually know the state, and g1a ðsÞ ¼ g1b ðsÞ ¼ g2a ðsÞ ¼ g2b ðsÞ ¼ 0. The point here is that if player 1 announces a and player 2 announces b, player 3 knows a deviation has occurred but can not find out who did deviate and what the true state is, and there is no appropriate way to continue the play. Consider the distribution on (Hy ; Hy ) induced by ðs1a ; s2b ; s3 Þ, and the ‘‘associated vector payo¤s’’. Since s2b must be not better than s2a if the state is a, ðs1a ; s2b ; s3 Þ must lead to a payo¤ of 0 for player 2 in state a. Similarly, since s1a must be not better than s1b if the state is b, ðs1a ; s2b ; s3 Þ must lead to a payo¤ of 0 for player 1 in state b. Hence a contradiction, since g2a þ g1b ¼ 1. Finally notice that the existence of uniform equilibrium in this game is not a problem: each of the players always playing the same mixed action (for example u, l and L) is an equilibrium. rr The variable introduced now is very important in repeated games with lack of information on one side. Given a fixed strategy s in S, the evolution of the uninformed player’s knowledge on the state of nature is described by a martingale of a posteriori (see for example [Ha]). For any t b 0 and ht A Ht , put for any k in K: ptk ðs; ht Þ ¼ Pp0 ; s ðkjht Þ ¼
p0k Ps k ðht Þ : Pp0 ; s ðht Þ
Put now pt ðs; ht Þ ¼ ðptk ðs; ht ÞÞk A K A DðKÞ (and define arbitrarily pt ðs; ht Þ in DðKÞ if Pp0 ; s ðht Þ ¼ 0). pt ðs; ht Þ is, for player 3, the conditional probability on the states of nature given that ðs1 ; s2 Þ is played by the informed players and ht has occurred in the first t stages. Note that p0 ðs; h0 Þ is the initial probability p0 , and that pt ðs; ht Þ does not depend on s3 and on the action played by player 3 at stage t. With respect to Pp0 ; s , ðpt ðsÞÞtb0 is a ðHt Þtb0 martingale1 with values in DðKÞ. Since it is bounded, it converges almost surely to some random variable py ðsÞ defined on ðH y ; Hy Þ and with values in DðKÞ. 1 For simplicity of notations, Ht and Hy are also used as s-algebras on H y .
226
J. Renault
We first formalize notions of rational payo¤s adapted to the games we consider. 3. Rational payo¤s We first study here the standard notion of individual rationality in G y ðp0 Þ. Then, we introduce a notion of joint rationality for the informed players. Note that for s in S satisfying point ii) of definition 2.1, s is a uniform equilibrium if and only if the two following conditions holds: Ei A IN; Ek A K; lim sup sup gi;k T ðs i ; ti Þ a gik ðsÞ; T
lim sup
T
ti A Si
sup g3; T ðs 3 ; t3 Þ a g3 ðsÞ:
t3 A S 3
3.1. Individual rationality An individually rational payo¤ for player i can be seen as a payo¤ such that i has no incentive to deviate in an observable way from a strategy leading to this payo¤, since while being punished he obtains less. Denote, for any player i and state k, by vik the minmax payo¤ of player i in G k : vik ¼
x i A
min Q j0i
max gik ðxi ; x i Þ
DðSj Þ xi A DðSi Þ
For each p in DðKÞ, we define, as for the two-player case, u3 ðpÞ as player 3’s minmax in the ‘‘non-revealing game with probability p’’: X min max p k g3k ðx1 ; x2 ; x3 Þ u3 ðpÞ ¼ ðx1 ; x2 Þ A DðS1 ÞDðS2 Þ x3 A DðS3 Þ
kAK
u3 is a continuous function from DðKÞ to R. Let vex u3 denote the (pointwise) greatest convex function f from DðKÞ to R such that f ð pÞ a u3 ðpÞ for all p in DðKÞ. Finally, we put M3 ¼ maxs A S; k A K jg3k ðsÞj. Proposition 3.1. a) Individual rationality for the informed players: For any i in IN and gi ¼ ðgik Þk A K in RK , gik b vik Ek A K , bs i A S i s:t: Ek A K; lim sup T
sup gi;k T ðsÞ a gik :
si A S i
b) Individual rationality for the uninformed player: For any g3 in R, g3 b vex u3 ð p0 Þ , bs 3 A S 3 s:t: lim sup sup gi; T ðsÞ a g3 : T
s3 A S 3
3-player repeated games with lack of information on one side
227
The idea of the individual rationality (IR) condition is simple for the informed players. An informed player i can be punished by the other players at the level vik if k is the true state: the other informed player can just announce the true state to the uninformed player before punishing. And player i can always defend vik if k is the true state by only considering the game G k . The IR condition for the uninformed player is a generalization of the condition for the two-player case: the informed players can ‘‘convexify’’ player 3’s non-revealing payo¤ function by changing in an appropriate way the a posteriori of the uninformed player. Proof of proposition 3.1: a) Let i be in IN and gi ¼ ðgik Þk A K be in RK . If gik b vik for all k, define s i k in the following way. Fix, for any k in K, x3 i in DðS3 i Þ and x3k in DðS3 Þ such k k k k that for all xi in DðSi Þ, gi ðxi ; x3 i ; x3 Þ a vi . Since player 3 i is informed, define s3 i as: observe the true state of nature k, then announce k to player 3 (using a code on the first actions of player 3 i), and finally play at each stage k . s3 is now just: deduce from the first actions of player the mixed action x3 i 3 i the value of k, and then play at each stage the mixed action x3k . It is clear that no strategy of player i can give him more than vik (and thus more than gik ) for some k. Conversely, assume now that gik < vik for some k. Since player i perfectly monitors all the actions played, he can defend vik by playing at each stage a best response in G k against the mixed action to be played by players i (i.e. in Nnfig). b) The proof given here is an adaptation of the proof of Aumann and Maschler [Au-Ma] for two players (and easily extends to the n-player case mentioned in remark 2.2). Its new aspect comes from the di¤erence between the sets DðS1 Þ DðS2 Þ and DðS1 S2 Þ. To see that player 3 can be forced down to vex u3 ðp0 Þ (and then that ) holds), one can just apply the splitting procedure of Aumann and Maschler: PM assume that p0 ¼ m¼1 lm pm for some positive M, with for each m, lm > 0, PM pm A DðKÞ and m¼1 lm ¼ 1. Define s1 as follows: first observe the state of nature k and choose m in f1; . . . ; Mg according to the probability lm pmk =p0k . Then, announce the chosen m to player 2 using a code on first actions, and finally play at each stage some mixed action x1; m in DðS1 Þ. The fact that player 3 also observes m has no influence here. For player 2, define s2 as: first observe the selected m, and then repeat forever some mixed action x2; m in DðS2 Þ. If for each m, ðx1; m ; x2; m Þ realizes the minimum in the expression of u3 ðpm Þ we will have for any s3 in S3 and T large enough: ~
Ep0 ; s ðg3k ð~ sT ÞÞ ¼
M X
Pp0 ; s ðmÞ
m¼1
a
M X
X
Pp0 ; s ðkjmÞEp0 ; s ðg3k ð~ sT Þ j k; mÞ
kAK
lm u3 ðpm Þ;
m¼1
since PP0 ; s ðmÞ ¼ lm and Pp0 ; s ðkjmÞ ¼ pmk for any m and k. For a good choice ~ M M and ðpm Þm¼1 , we will then get: Ep0 ; s ðg3k ð~ sT ÞÞ a vex u3 ðp0 Þ for T of ðlm Þm¼1 large enough.
228
J. Renault
To conclude the proof, we have to show2 that the uninformed player can defend vex u3 ðp0 Þ. Let ðs1 ; s2 Þ be in S1 S2 , and define s3 as follows: put, for any t b 0 and ht A Ht , X sðht Þ ¼ ptk ðs; ht Þs1k ðht Þ n s2k ðht Þ A DðS1 S2 Þ; kAK
where n is used to denote a product probability measure. For player 3 (knowing that ðs1 ; s2 Þ is played by players 1 and 2), after ht has occurred, sðht Þ represents the distribution on the next actions of the informed players: Eðs1 ; s2 Þ A S1 S2 , Pp0 ; s ððs1 ; s2 Þ is played at stage t þ 1 j ht Þ ¼ sðht Þðs1 ; s2 Þ. We define s3 ðht Þ such that the uninformed player just plays a best response according to his current knowledge: X ptk ðs; ht Þg3k ðs1k ðht Þ; s2k ðht Þ; x3 Þ: s3 ðht Þ A Argmax x3 A DðS3 Þ k A K
Assume finally that s ¼ ðs1 ; s2 ; s3 Þ is played. All following probabilities and expectations are induced by Pp0 ; s . For simplicity, the symbol s is omitted in the notation pt ðs; ht Þ. The following expressions (1) and (2) were already used for the two-player case ([Au-Ma]). The proofs perfectly extend to our case, and are given in the Appendix for the sake of completeness.
ET b 1;
T 1 X 1X Ek ptþ1 pt k a T t¼0 kAK
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p0k ð1 p0k Þ : T
ð1Þ
Et b 0, Eht A Ht , ~
~
Eðk ptþ1 pt k j ht Þ ¼ Eðks1k ðht Þ n s2k ðht Þ sðht Þk j ht Þ:
ð2Þ
We now know that the di¤erence between the lotteries actually played ~ ~ s1k ðht Þ n s2k ðht Þ in DðS1 Þ DðS2 Þ and the approximation sðht Þ in DðS1 S2 Þ will be small enough. To conclude, we have to introduce some k0 in K depending only on ht such that s1k0 ðht Þ n s2k0 ðht Þ is close to s1k ðht Þ n s2k ðht Þ for any k having a significant probability to be the true state. This will show that player 3 is almost facing a product probability independent of the state of nature and we will be able to introduce the function u3 . Fix t b 0 and ht in Ht . We first have: ~
Eðg3k ð~ stþ1 Þ j ht Þ ¼
X
ptk ðht Þg3k ðs1k ðht Þ; s2k ðht Þ; s3 ðht ÞÞ
kAK
b
X
ptk ðht Þg3k ðs1k ðht Þ; s2k ðht Þ; x3 Þ
Ex3 A DðS3 Þ:
kAK
Simply fix now k0 in K minimizing fks1k ðht Þ n s2k ðht Þ sðht Þk : k A Kg. We have for all x3 : 2 I wish to thank S. Sorin for suggesting this kind of proof.
3-player repeated games with lack of information on one side
229
X
ptk ðht Þðg3k ðs1k ðht Þ; s2k ðht Þ; x3 Þ g3k ðs1k0 ðht Þ; s2k0 ðht Þ; x3 ÞÞ
kAK
a
X
ptk ðht ÞM3
kAK
X
js1k ðht Þðs1 Þ s2k ðht Þðs2 Þ x3 ðs3 Þ
ðs1 ; s2 ; s3 Þ A S
s1k0 ðht Þðs1 Þ s2k0 ðht Þðs2 Þ x3 ðs3 Þj a M3
X
ptk ðht Þks1k ðht Þ n s2k ðht Þ s1k0 ðht Þ n s2k0 ðht Þk
kAK
a M3
X
ptk ðht Þðks1k ðht Þ n s2k ðht Þ sðht Þk
kAK
þ ksðht Þ s1k0 ðht Þ n s2k0 ðht ÞkÞ ~
~
a 2M3 Eðks1k ðht Þ n s2k ðht Þ sðht Þk j ht Þ: ~
stþ1 Þ j ht Þ b u3 ð pt ðht ÞÞ We then obtain, by definition of u3 : Eðg3k ð~ ~ ~ 2M3 Eðks1k ðht Þ n s2k ðht Þ sðht Þkj ht Þ. 1 P T 1 k~ 1 P T 1 P Hence, for all T b 1, Ep0 ; s g ð~ s Þ b tþ1 t¼0 ht A Ht Pðht Þ 3 T T t¼0 ~ ~ ðu3 ðpt ðht ÞÞ 2M3 Eðks1k ðht Þ n s2k ðht Þ sðht Þk j ht ÞÞ. Using (1), (2) and the convexity of vex u3 , we finally get: ET b 1;
M3 X g3; T ðsÞ b vex u3 ð p0 Þ 2 pffiffiffiffi T kAK
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p0k ð1 p0k Þ:
rrrr
3.2. Joint rationality for the informed players The motivation for the study of joint rationality comes from example 2.3. The point is that during the game, the uninformed player may see that a deviation has occurred without knowing which of the informed players did deviate. We then have to consider simultaneous punishments of the informed players (this idea of punishing a whole class of players simultaneously can also be found in [To]). More precisely, we will often consider situations where player 3 knows that either ‘‘player 1 has deviated and the true state is k’’ or ‘‘player 2 has deviated and the true state if k 0 ’’. We will then look for plays giving low payo¤s both for player 1 in state k and for player 2 in state k 0 . For any couple of states ðk; k 0 Þ, we thus put: JR1; 2 ðk; k 0 Þ ¼ fðg1 ; g2 Þ A RK RK : bz A DðSÞ s:t: g1k ðzÞ a g1k and 0
0
g2k ðzÞ a g2k g: Put also, for k in K,
230
J. Renault
IR1 ðkÞ ¼ fðg1 ; g2 Þ A RK RK : g1k b v1k g; IR 2 ðkÞ ¼ fðg1 ; g2 Þ A RK RK : g2k b v2k g: Set finally JR1; 2 ¼ 7ðk; k 0 Þ A KK JR1; 2 ðk; k 0 Þ, IR1 ¼ 7k A K IR1 ðkÞ, and IR 2 ¼ 7k A K IR 2 ðkÞ. Proposition 3.2. Joint rationality for the informed players: For any g1; 2 ¼ ðg1k ; g2k Þk A K in RK RK , g1; 2 A IR1 X IR 2 X JR1; 2 , bs A S s:t: Ei A IN; Ek A K; lim sup T
sup gi;k T ðs i ; ti Þ a gik :
ti A S i
Proof: ) Assume that g1; 2 belongs to IR1 X IR 2 X JR1; 2 . For each pair of states ðk; k 0 Þ in K K, fix an infinite sequence of pure actions (elements of S) with 0 an average corresponding to a frequency z k; k in DðSÞ such that: 0
0
g1k ðz k; k Þ a g1k
0
and
0
g2k ðz k; k Þ a g2k :
We define s in S as follows: – first, each of the informed players announces the selected state (using a code on his actions). – then, if player 1 announced k and player 2 announced k 0 , the three players 0 play the path of pure actions corresponding to the frequency z k; k . – if one of the informed players deviates from this path, the other players just 00 punish him forever in order to achieve his minmax in the game G k where 00 k has been announced by the other informed player. The idea of this strategy is very simple: if player 1 announces k and player 2 announces k 0 distinct from k, it is clear that a deviation has occurred but both informed players can be suspected. In this case, the progress of the game 0 has to be such that it punishes player 1 in the game G k and simultaneously k player 2 in the game G , since if one informed player deviates the true state of nature is announced by the other informed player. Since g1; 2 is in IR1 X IR 2 , no deviation of an informed0 player from a path of pure actions is profitable. Hence, by definition of ðz k; k Þðk; k 0 Þ A KK , it is also the case for deviations concerning the announcement of the selected state. It is thus easy to see that s satisfies the required condition. ( By contraposition. Let g1; 2 in RK RK be such that g1; 2 B IR1 X IR 2 X JR1; 2 . If g1; 2 B IR1 or g1; 2 B IR 2 , the result is clear (and just a consequence of proposition 3.1). We thus assume that g1; 2 B JR1; 2 ðk; k 0 Þ for some ðk; k 0 Þ in 00 00 0 K K, and consider s ¼ ððs1k Þk 00 A K ; ðs2k Þk 00 A K ; s3 Þ in S. ðs1k ; s2k ; s3 Þ induces 0 a probability distribution on ðHy ; Hy Þ. The idea is that if ðs1k ; s2k ; s3 Þ is played, the uninformed player can not know whether player 1 is deviating
3-player repeated games with lack of information on one side
231
from s1 in state k or player 2 is deviating from s2 in state k 0 . Consider now, for any T b 1: ! ! T T 1X 1 X 0 k k g ð~ st Þ and bT ¼ Es k 0 ; s k ; s3 g ð~ st Þ : aT ¼ E s k 0 ; s k ; s 3 1 2 1 2 T t¼1 1 T t¼1 2 For all0 T, ðaT ; bT Þ belongs to the compact convex set of R 2 fðg1k ðzÞ; g2k ðzÞÞ : z A DðSÞg. By assumption, the mapping which associates to 0 0 any z in DðSÞ the maximum of g1k ðzÞ g1k and g2k ðzÞ g2k has positive values. By continuity, one can find e > 0 such that for any positive T, a0 T b g1k þ e k0 or b T b g2 þ e. Hence limT sup aT > g1k or limT sup b T > g2k . Now, if 0 limT sup aT > g1k , then player 1 by playing s1k in state k will obtain a (lim sup) 0 payo¤ greater than g1k . If limT sup bT > g2k , considering the use of s2k by rrrr player 2 in state k 0 ends the proof. Put now IR3 ¼ fg3 A R : g3 b vex u3 ðp0 Þg, and denote by F the set of feasible payo¤s: F ¼ ððg1k Þk A K ; ðg2k Þk A K ; g3 Þ A RK RK R : bðz k Þk A K with Ek A K; z k A DðSÞ; g1k ¼ g1k ðz k Þ; g2k ¼ g2k ðz k Þ and g3 ¼
X
p0k g3k ðz k Þ :
kAK
It is clear that any equilibrium payo¤ must belong to F, since for any s in S and positive T, ððg1;k T Þk A K ; ðg2;k T Þk A K ; g3; T Þ belongs to the convex compact set F. Using propositions 3.1 and 3.2, and the characterization of uniform equilibria given at the beginning of this section, we more precisely have: Claim 3.3. If ðg1 ; g2 ; g3 Þ in RK RK R is an equilibrium payo¤ of G y ðp0 Þ, then ðg1 ; g2 ; g3 Þ A F , ðg1 ; g2 Þ A IR1 X IR 2 X JR1; 2 , and g3 A IR3 . We now study the simplest kind of equilibria, where the uninformed player eventually learns the state. 4. Completely revealing equilibria Given any strategy s in S, py ðsÞ is the random variable on ðH y ; Hy Þ with values in DðKÞ representing player 3’s final knowledge on the true state of nature. Denote for any state k by dk the Dirac measure on k, i.e. the element of DðKÞ such that dkk ¼ 1. Definition 4.1. s A S is a completely revealing equilibrium (CR equilibrium) if: (i) s is a uniform equilibrium; (ii) Ek A K; py ðsÞ ¼ dk
Ps k -almost surely:
Remark 4.2: A stronger definition of completely revealing equilibrium can be given by replacing (ii) by the following condition, which says that all the in-
232
J. Renault
formation has been revealed to the uninformed player after a fixed number of stages: bT b 0; Ek A K;
pT ðsÞ ¼ dk
Ps k -a:s:
The proof of the following proposition however shows that both definitions give the same set of CR equilibrium payo¤s. rr We now characterize the payo¤s of CR equilibria, and hence games where CR equilibria exist. Proposition 4.3. For any g ¼ ððg1k Þk A K ; ðg2k Þk A K ; g3 Þ in RK RK R, g is a CR equilibrium payo¤ , ððg1k Þk A K ; ðg2k Þk A K Þ A IR1 X IR 2 X JR1; 2 and there exists ðz k Þk A K A ðDðSÞÞ K such that: i) g1k ¼ g1k ðz k Þ; g2k ¼ g2k ðz k Þ Ek A K; X ii) g3 ¼ p0k g3k ðz k Þ; and g3k ðz k Þ b v3k
Ek A K:
kAK
Proof: ) Let s be a CR equilibrium with payo¤ ððg1k Þk A K ; ðg2k Þk A K ; g3 Þ. By claim 3.3, ððg1k Þk A K ; ðg2k Þk A K Þ belongs to IR1 X IR 2 X JR1; 2 . For all positive T, ððg1;k T ðsÞÞk A K ; ðg2;k T ðsÞÞk A K ; ðg3;k T ðsÞÞk A K Þ belongs to the compact set F 0 ¼ fððg1k ðz k ÞÞk A K ; ðg2k ðz k ÞÞk A K ; ðg3k ðz k ÞÞk A K Þ : Ek; z k A DðSÞg H RK RK RK . Since in definition 2.1, ððg3;k T ðsÞÞk ÞTb1 is not assumed to converge as T goes to infinity, we consider a convergent subsequence ððg1;k cðTÞ ðsÞÞk , ðg2;k cðTÞ ðsÞÞk ; ðg3;k cðTÞ ðsÞÞk ÞTb1 with limit ððg1k Þk ; ðg2k Þk ; ðg3k Þk Þ in F 0 . This the existence of ðz k Þk A K s.t.: Ei A N; Ek A K; gik ¼ gik ðz k Þ and P gives k k g 3 ¼ k A K p0 g 3 . It thus remains to show that g3k b v3k for all k (this part of the proof can extend to an arbitrary number of informed players). Assume by contradiction that for some k0 in K we have g3k0 < v3k0 . We construct a profitable deviation of player 3 from s. Fix e > 0, and define for any n > 0: A n ¼ fhy ¼ ðst Þtb1 A Hy : bT b 0; pTk0 ðs; ðs1 ; . . . ; sT ÞÞ b 1 1=ng A Hy : s being completely revealing, we have for any k 0 k0 , Ps k ð7n>0 A n Þ ¼ 0. Hence one can fix n0 > 0 such that Ps k ðA n0 Þ a e for any k 0 k0 . We define t3 in S3 as follows: – as long as the a posteriori probability of state k0 is lower than 1 1=n0 , play according to s3 . – as soon as hT such that pTk0 ðs; hT Þ b 1 1=n0 occurs, consider only state k0 and play against s1k0 and s2k0 in order to obtain at each stage an expected payo¤ not less than v3k0 if the state is k0 .
3-player repeated games with lack of information on one side
233
To show that t3 is (for a good choice of e) profitable for player 3, we put for any T b 0: CTn0 ¼ fhy ¼ ðst Þtb1 A Hy : bt A f0; . . . ; Tg; ptk0 ðs; ðs1 ; . . . ; st ÞÞ b 1 1=n0 g A HT ; and denote by Hy nCTn0 the complement of CTn0 in Hy . By assumption, Ps k0 ð6Tb0 CTn0 Þ ¼ 1, hence one can find T0 b 0 such that Ps k0 ðCTn0 Þ b 1 e for any T b T0 . We have for all T, CTn0 H A n0 and, for any state k, Ps k ; s k ; s3 ðCTn0 Þ ¼ Ps k ; s k ; t3 ðCTn0 Þ. 1 2 1 2 P ~ sTþ1 ÞÞ ¼ k A K p0k Es k ; s k ; t3 Fix finally T b T0 . We have Es3 ; t3 ðg3k ð~ 1 2 sTþ1 ÞÞ. For simplicity, we put in the following computations, for any state ðg3k ð~ 0 0 k, Pk ¼ Ps k ; s k ; s3 , Pk ¼ Ps k ; s k ; t3 , Ek ¼ Es k ; s k ; s3 , and Ek ¼ Es k ; s k ; t3 . 1 2 1 2 1 2 1 2 First, Ek0 0 ðg3k0 ð~ sTþ1 ÞÞ ¼ Pk0 0 ðCTn0 ÞEk0 0 ðg3k0 ð~ sTþ1 Þ j CTn0 Þ þ Pk0 0 ðHy nCTn0 Þ Ek0 0 ðg3k0 ð~ sTþ1 Þ j Hy nCTn0 Þ. Since Pk0 0 ðCTn0 Þ b 1 e and by construction sTþ1 Þ j CTn0 Þ b v3k0 , we get: Ek0 0 ðg3k0 ð~ sTþ1 ÞÞ b ð1 eÞv3k0 2eM3 : Ek0 0 ðg3k0 ð~ sTþ1 ÞÞ ¼ Pk0 ðCTn0 ÞEk0 ðg3k ð~ sTþ1 Þ j CTn0 Þ þ Secondly, for k 0 k0 , Ek0 ðg3k ð~ n0 n0 n0 n0 0 0 k k Pk ðHy nCT ÞEk ðg3 ð~ sTþ1 Þ j Hy nCT Þ þ Pk ðCT ÞEk ðg3 ð~ sTþ1 Þ j CT Þ Pk ðCTn0 Þ n n Ek ðg3k ð~ sTþ1 Þ j CT0 Þ. Since Ek0 ðg3k ð~ sTþ1 Þ j Hy nCT0 Þ ¼ Ek ðg3k ð~ sTþ1 Þ j Hy nCTn0 Þ n n and Pk0 ðCT0 Þ ¼ Pk ðCT0 Þ a Pk ðA n0 Þ a e, we get: sTþ1 ÞÞ b Ek ðg3k ð~ sTþ1 ÞÞ 2eM3 : Ek0 ðg3k ð~ ~
P
sTþ1 ÞÞ b p0k0 ðð1 eÞv3k0 2eM3 Þ þ Thus we obtain, for T b T0 : Es3 ; t3 ðg3k ð~ P k0 k0 k k sTþ1 ÞÞ 2eM3 Þ b p0 v3 þ k0k0 p0k Ek ðg3k ð~ sTþ1 ÞÞ 3eM3 . k0k0 p0 ðEk ðg3 ð~ Taking the stage-average and the lim sup as T goes to infinity gives: lim sup g3; T ðs3 ; t3 Þ b 3eM3 þ p0k0 v3k0 þ T
X
p0k g3k :
k0k0
k0 k0 Since we assumed P k kg3 < v3 , for e low enough one can have limT sup g3; T ðs3 ; t3 Þ > k p0 g3 ¼ g3 . This contradicts the equilibrium condition for s and ends the first part of the proof. ( This is the easy part of the proof. For any pair ðk; k 0 Þ with k 0 k 0 , let 0 0 0 0 k; k 0 z in DðSÞ be such that g1k ðz k; k Þ a g1k and g2k ðz k; k Þ a g2k . For any k, put k; k k also z ¼ z . The idea of the equilibrium strategy is the same as in the proof of proposition 3.2 (part )). Define s in S as follows:
– first, each of the informed players announces the selected state using a code on his actions. – then, if player 1 announced k and player 2 announced k 0 , the three players 0 play an infinite sequence of pure actions leading to the frequency z k; k in DðSÞ. – if one of the players deviates from this path of pure actions, the other players
234
J. Renault
just punish him forever in order to achieve his minmax in the game G k where k has been announced by an informed player who did not deviate from the path of pure actions. For any player i and state k, limT!y gi;k T ðsÞ ¼ gik ðz k Þ, hence g is the payo¤ induced by s. Moreover, it is clear that s is a CR equilibrium. rrrr Notice that in remark 2.2, we actually proved the existence of a CR equilibrium in any n-player repeated game with lack of information on one side (and perfect observation) for n b 4. This is no longer the case here, as shown by example 2.3. We then have to consider other kinds of equilibria. 5. Joint plans The initial definition given in [Au-Ma] for two players can be generalized here in several ways. We have chosen the simplest one: Definition 5.1. For i in IN, a joint plan for player i is a tuple ðM; l; P; z; gi Þ where: – M is a non empty finite set of messages (or signals). k – l ¼ ðl k Þk A K (‘‘signalling P k kstrategy’’) with for each state k, l A DðMÞ and Em A M, lm ¼def k p0 lm > 0. – P ¼ ð pm Þm A M such that Em, pm A DðKÞ (a posteriori on K given m) with pmk ¼ p0k lmk =lm Ek A K. – z ¼ ðzm Þm A M (‘‘contract’’) with Em A M, zm A DðSÞ. – gi A RK with Ek A K, gik ¼ maxm A M gik ðzm Þ. As usual, for convenience joint plans are defined as quantities rather than as strategies of the repeated game. However, the idea is to consider an underlying strategy as follows. Player i observes the true state k, then chooses m according to l k and announces m to the other players using a code on his actions. Then, pm is the a posteriori of player 3 on the state of nature. Finally, the three players are supposed to play an infinite sequence of pure actions leading to the frequency zm . Concerning punishments, if one of the informed players (say j A f1; 2g) deviates from the path leading to zm , the other informed player has to announce the state of nature k, and then the players in Nnf jg punish player j forever to his minmax payo¤ in G k . If player i’s announcement is ‘‘out of any code’’, he is similarly punished to his minmax payo¤ in G k . If the uninformed player deviates from the path to zm , the informed players punish him to 3.1 (part b). vex u3 ð pm Þ, using P Pa strategy given by proposition l ¼ 1 and p ¼ lm pm . Conversely, Note that m 0 m A M m A M P if p0 ¼ P m A M lm pm with M finite, for each m lm > 0, pm A DðKÞ and m lm ¼ 1, then one can find a signalling strategy l ¼ ðl k Þk A K giving P ¼ ð pm Þm A M as a posteriori: just put, for any m in M and k in K, lmk ¼ lm pmk =p0k . Definition 5.2. For i in IN, a joint plan for player i ðM; l; P; z; gi Þ is an equilibrium joint plan (EJP) for i if: i) Ek A K, Em A M with lmk > 0, g1k ðzm Þ b v1k and g2k ðzm Þ b v2k ,
3-player repeated games with lack of information on one side
235
P ii) Em A M, k A K pmk g3k ðzm Þ b vex u3 ðpm Þ, iii) Ek A K, Em A M with lmk > 0, gik ðzm Þ ¼ gik . i) is a condition of individual rationality for the informed players, whereas ii) is individual rationality for the uninformed player. iii) is an incentive condition for player i to announce the message according to the signalling strategy. Lemma 5.3. Given an equilibrium joint plan for i ðM; l; P; z; gi Þ, one can construct a uniform equilibrium with payo¤ gi for player i. Moreover, if we put for k k any k, g3i ¼ minfg3i ðzm Þ : m A M; pmk > 0g, then ðg1 ; g2 Þ A IR1 X IR 2 X JR1; 2 . Proof: Denote by s the underlying strategy described after definition 5.1. We simply show that it is a uniform equilibrium. If the true state is k, player i’s payo¤ induced by s k is gik , since gik ðzm Þ ¼ gik whenever m has a positive probability to be chosen in state k. Since gik ¼ maxfvik ; maxfgik ðzm Þ : m A Mgg, no deviation can give him more than gik . For k k ðzm Þ b v3i the other informed player, no deviation can be profitable since g3i whenever m can be chosen with positive probability in state k. Finally notice that for any t3 in S3 and positive T, ! ! T T X X 1X 1X k k~ k k Ep0 ; s3 ; t3 g ð~ st Þ ¼ p0 lm Es k ; t3 g ð~ st Þ j m 3 T t¼1 3 T t¼1 3 mAM kAK ¼
X
lm
mAM
X kAK
pmk Es k ; t3 3
! T 1X k g ð~ st Þ j m : T t¼1 3
If m has been announced, the a posteriori of player 3 is pm . Hence for any positive e, one can find T0 such that for T b T0 : ! T X X 1X k~ Ep0 ; s3 ; t3 g3 ð~ st Þ a lm max vex u3 ð pm Þ; pmk g3k ðzm Þ þ e T t¼1 mAM kAK a
X
lm
mAM
X
pmk g3k ðzm Þ þ e
kAK
a lim g3; T 0 ðsÞ þ e: 0 T !y
Thus, player 3 also has no profitable deviation, and s is a uniform equilibrium (note that conditions i), ii) and iii) are also necessary for s to be an equilibr rium). The fact that ðg1 ; g2 Þ A IR1 X IR 2 X JR1; 2 is straightforward. 6. The case of two states of nature In this section, we show the existence of uniform equilibria in the case of two states of nature. More precisely, we have the following result. Theorem 6.1. Assume that jKj ¼ 2. i) There exists a completely revealing equilibrium or an equilibrium joint plan.
236
J. Renault
ii) Equilibrium joint plans may fail to exist. Completely revealing equilibria and equilibrium joint plans for player 1 may fail to exist. Completely revealing equilibria and equilibrium joint plans for player 2 may fail to exist. Proof: i) Assume that K ¼ fa; bg. If there exists x in DðSÞ such that g1a ðxÞ a v1a and g2b ðxÞ a v2b and also x 0 in DðSÞ such that g1b ðx 0 Þ a v1b and g2a ðx 0 Þ a v2a then IR1 X IR 2 H JR1; 2 ða; bÞ X JR1; 2 ðb; aÞ. Considering, for any k in K, z k in DðSÞ such that gik ðz k Þ b vik for all i in N gives the existence of some CR equilibrium (see proposition 4.3). We thus assume w.l.o.g. that for any x A DðSÞ; g1a ðxÞ > v1a or g2b ðxÞ > v2b . We view DðKÞ as fð p a ; p b Þ A R 2 : p a b 0; p b b 0; p a þ p b ¼ 1g. Recall that p0a K K and p0b have been assumed tobe positive. R R is viewed as the Euclidean j1a j2a , withjik A R fori A f1; 2g and k A K. space of 2 2 matrices j ¼ j1b j2b b0 0 We also simple notations such as , just meaning here introduce >0 j1a j2a a a b : j1 b 0; j2 ¼ 0; j2 > 0 . Consider now the following corj¼ j1b j2b respondence: F : DðKÞ ! RK RK ( g1a ðxÞ v1a p 7! g1b ðxÞ v1b
) X g2a ðxÞ v2a p k g3k ðxÞ b u3 ðpÞ : x A DðSÞ; g2b ðxÞ v2b kAK
The consideration, for all p in DðKÞ, k and k 0 in K, of some (mixed) equik k0 for player 1, g for player libriumPof the00 finite game with payo¤s given by g 2 1 00 2 and k 00 p k g3k for player 3, first gives: 0 (1) Ep A DðKÞ; Ek; k 0 A K; bj A Fð pÞ s.t. j1k b 0 and j2k b 0. We clearly also have: (2) F has a compact graph and non empty convex values. a b (3) Ep A DðKÞ; Ej A FðpÞ, j1 > 0 or j2 > 0. b0 b0 If Fð p0 Þ X 0 q, a very simple equilibrium joint plan with a b0 b0 zm0 in DðSÞ such that gik ðzm0 Þ b vik single message m0 exists: P just define k k Ei A f1; 2g, Ek A K and k A K p0 g3 ðzm0 Þ b u3 ðp0 Þ ðb vex u3 ð p0 ÞÞ. Note that this joint plan, called non revealing because the unique a posteriori is the a priori p0 , is an equilibrium joint plan for both of the informed players. b0 b0 We thus assumenow that Fðp0 Þ X b0 b0 ¼ q. The set p A DðKÞ : b0 b0 Fð pÞ X ¼ q is an open set of DðKÞ. Let R be the connected b0 b0 component of this set that contains p0 . Put also: > 0 b0 R1 ¼ p A R : FðpÞ X 0q and b0 < 0 < 0 b0 0q : R 2 ¼ p A R : FðpÞ X b0 > 0
3-player repeated games with lack of information on one side
For any p in R, we have Fð pÞ X
b0
237
0 q by (1), hence by (3) p A b0 R1 W R 2 . We then have R ¼ R1 W R 2 , and assume w.l.o.g. that p0 A R1 . We now prove that R1 and R 2 are two disjoint closed subsets of the connected set R to conclude that R 2 ¼ q. In order to prove that R1 is closed in R, let ð pn Þn A N be a sequence of elements of R 1 converging to some p in R. For any n, let jn in Fðpn Þ be such > 0 b0 that jn A . Taking a converging subsequence of ðjn Þn A N gives b0 < 0 > 0 b0 the existence of some j in FðpÞ such that j A b0 a0 . Since p A R, j A > 0 b0 R 2 ) is closed and p A R1 . So R1 (and similarly in R. b0 < 0 > 0 b0 Finally,if p in DðKÞ is such that Fð pÞ X b0 < 0 0 q and FðpÞ X < 0 b0 0 b0 0 q, by convexity of Fð pÞ one has Fð pÞ X 0 q, b0 > 0 b0 and by (3) p can not belong3 to R. Hence R1 X R 2 ¼ q. We then obtain R ¼R1 . Now,for any p in R, we have the existence > 0 b0 0 of some jð pÞ A Fð pÞ X b0 < 0 and, by (1), of some j ðpÞ A FðpÞ X . Hence by convexity of FðpÞ we also have some j 00 ðpÞ in FðpÞ X b0 b0 >0 <0 . Taking another convex combination of jðpÞ and j 00 ðpÞ now b0 0 > 0 0 gives the existence of some j 000 ð pÞ in FðpÞ X . b0 < 0 Let now f p1 ; p2 g be the (relative) frontier of R: p1a ¼ supfp a : p A Rg
and
p2b ¼ supf p b : p A Rg
00 By compactness of the graph of F, we havethe existence of some j ðp1 Þ A b0 a0 b0 0 Fð p1 Þ X , and j 000 ðp1 Þ A Fðp1 Þ X . Now, if p1 B R, b0 0 b0 a0 b0 b0 then Fð p1 Þ X 0 q. Using of Fð p1 Þ, we have the the convexity b0 b0 b0 0 existenceof some j^ðp1 Þ in Fðp1 Þ X . Let z1 be in DðSÞ such that 0 g1a ðz1 Þ v1a g2a ðz1 Þ v2a b0 P k k and j^ðp1 Þ ¼ k A K p1 g3 ðz1 Þ b u3 ð p1 Þ. Otherg1b ðz1 Þ v1b g2b ðz1 Þ v2b wise, p1 belongs to R which is open in DðKÞ, hence p1 ¼ da . Let z1 be in DðSÞ a a a a g ðz Þ v g ðz Þ v 1 1 1 2 1 2 and g3a ðz1 Þ b u3 ðda Þ. such that j 000 ð p1 Þ ¼ g1b ðz1 Þ v1b g2b ðz1 Þ v2b Wecan proceed similarly for p2 . If p2 B R, let z2 in DðSÞ be such g1a ðz2 Þ v1a g2a ðz2 Þ v2a b0 0 that belongs to Fð p2 Þ X and g1b ðz2 Þ v1b g2b ðz2 Þ v2b b0 0 P k k k A K p2 g3 ðz2 Þ b u3 ðp2 Þ. If p2 A R (then p2 ¼ db ), let z2 be such that
3 This argument can not be generalized to the case of more states.
238
J. Renault
g1a ðz2 Þ v1a g2a ðz2 Þ v2a b0 a0 belongs to Fðp Þ X and g3b ðz2 Þ b 2 g1b ðz2 Þ v1b g2b ðz2 Þ v2b b0 0 u3 ðdb Þ. Let now l1 and l2 be positive numbers such that l1 þ l2 ¼ 1 and p0 ¼ l1 p1 þ l2 p2 . Consider the joint plan ðM; l; P; z; g2 Þ where:
– – – – –
M ¼ f1; 2g, l ¼ ðl a ; l b Þ, with l1a ¼ l1 p1a =p0a , l1b ¼ l1 p1b =p0b , P ¼ f p1 ; p2 g, z ¼ ðz1 ; z2 Þ, g2 ¼ ðg2a ; g2b Þ, with g2a ¼ g2a ðz1 Þ and g2b ¼ g2b ðz2 Þ.
One can easily check that ðM; l; P; z; g2 Þ is an equilibrium joint plan for player 2 (if p0 A R 2 , we would obtain an equilibrium joint plan for the first player). ii) The following example, with the same notations as in example 2.3, first shows that joint plan equilibria may fail to exist. p0 ¼ ð1=2; 1=2Þ l
l
K ¼ fa; bg; r
u
ð1; 2; 0Þ
ð0; 0; 0Þ
u
ð0; 0; 1Þ
ð0; 0; 1Þ
d
ð1; 2; 0Þ
ð0; 0; 0Þ
d
ð2; 0; 1Þ
ð2; 0; 1Þ
L
state a
l
r
u
ð0; 0; 1Þ
ð0; 2; 1Þ
d
ð0; 0; 1Þ
ð0; 2; 1Þ L
r
R l
r
u
ð2; 1; 0Þ
ð2; 1; 0Þ
d
ð0; 0; 0Þ
ð0; 0; 0Þ
state b
R
First note that the payo¤s of player 3 only depend on his action and on the state. Thus for all p in DðKÞ, u3 ð pÞ ¼ vex u3 ðpÞ ¼ maxf p a ; p b g. Note also that v1a ¼ v1b ¼ v2a ¼ v2b ¼ 0. g1a ðzÞ v1a g2a ðzÞ v2a For any z in DðSÞ, belongs to the convex hull g1b ðzÞ v1b g2b ðzÞ v2b 1 2 0 0 0 0 2 0 of ; ; ; which can be strictly sepa0 0 0 2 2 1 0 0 b0 b0 rated from . Hence, no p in DðKÞ such that p a > 0 and p b > 0 can b0 b0 be an a posteriori of an equilibrium joint plan (see i) of definition 5.2). We then only have to deal with joint plans ðM; l; P; z; gi Þ such that M ¼ f1; 2g, l1a ¼ 1, l2b ¼ 1, P ¼ f p1 ; p2 g with p1 ¼ da and p2 ¼ db . Consider now z1 . Player 3 has to play Rfor ii) of 5.2 to be satisfied, g1a ðz1 Þ v1a g2a ðz1 Þ v2a ¼ and then player 1 has to play u. This gives g1b ðz1 Þ v1b g2b ðz1 Þ v2b
3-player repeated games with lack of information on one side
239
a g1 ðz2 Þ v1a g2a ðz2 Þ v2a 1 2 0 0 ¼ . But in this . Similarly, g1b ðz2 Þ v1b g2b ðz2 Þ v2b 0 0 2 1 case player 1 would prefer to announce the message 2 in state a and player 2 would prefer to announce the message 1 in state b. Hence condition iii) of definition 5.2 can not be satisfied. We now give an example where no CR equilibrium and no equilibrium joint plan for player 2 exist. By symmetry of the informed players, this will conclude the proof. As usual, K ¼ fa; bg and p0 ¼ ð1=2; 1=2Þ.
l
r
u
ð0; 2; 0Þ
ð0; 2; 0Þ
d
ð1; 1; 0Þ
ð1; 1; 0Þ L
r
u
ð0; 0; 1Þ
ð0; 0; 1Þ
d
ð1; 0; 1Þ
ð1; 0; 1Þ
state a
l
r
u
ð1; 0; 3Þ
ð1; 0; 3Þ
d
ð0; 0; 3Þ
ð0; 1; 3Þ L
l
state b
R l
r
u
ð1; 1; 0Þ
ð1; 1; 0Þ
d
ð1; 0; 0Þ
ð1; 0; 0Þ R
Again, v1a ¼ v2a ¼ v1b ¼ v2b ¼ 0. Since player 3’s payo¤s only depend on his own action and on the state, u3 ð pÞ ¼ vex u3 ðpÞ ¼ maxf3p b ; p a g for all p. Finally notice that g1b ðzÞ þ g2a ðzÞ ¼ 1 Ez A DðSÞ. Assume that a CR equilibrium with payo¤ g ¼ ððg1k Þk A K ; ðg2k Þk A K ; g3 Þ exists. Then in state a, player 3 will play R (see point ii) of proposition 4.3) and player 1 will play u. In state b, player 3 will play L, so to be individually rational player 1 will play d and player 2 will play l. This gives g1a ¼ g2a ¼ g1b ¼ g2b ¼ 0. But then ððg1k Þk A K ; ðg2k Þk A K Þ does not belong to JR1; 2 ðb; aÞ, yielding a contradiction with proposition 4.3. Let now ðM; l;P P; z; g2 Þ be an equilibrium joint plan for player 2. Since p0 ¼ ð1=2; 1=2Þ ¼ m A M lm pm , one can find p1 and p2 in P such that p1a b 1=2 and p2b b 1=2. By point ii) of definition 5.2 and since 3 p2b > p2a , we must have that z2 is such that L is played by the uninformed player. Individual rationality for the informed players yields z2 ¼ ðd; l; LÞ, hence g2b ¼ 0. Since g1a ðz2 Þ < 0, we necessarily have p2 ¼ db . Now p1 ¼ da is not possible, as CR equilibria do not exist. Hence p1a > 0 and p1b > 0. The former inequality gives that z1 ¼ a1 ðu; l; LÞ þP a2 ðu; r; LÞ þ a3 ðu; l; RÞ þ a4 ðu; r; RÞ for some non nega4 tive a1 ; a2 ; a3 ; a4 with r¼1 ar ¼ 1. Using individual rationality for player 1 in state b gives a1 a2 þ a3 þ a4 b 0, hence a1 þ a2 a 1=2. But this induces g2b ðz1 Þ > 0, hence g2b > 0. Again, we have a contradiction (in state b, player 2 would prefer to induce z1 instead of z2 ). This ends the proof of theorem 6.1. rrrr To conclude with the last example of the proof, one can check that an equilibrium joint plan for player 1 is given by:
240
J. Renault
M ¼ f1; 2g, l ¼ ðl a ; l b Þ with l1a ¼ 1, l1b ¼ 1=2, hence P ¼ fp1 ; p2 g, with p1 ¼ ð2=3; 1=3Þ, p2 ¼ ð0; 1Þ, z ¼ ðz1 ; z2 Þ with z1 ¼ a1 ðu; l; LÞ þ a2 ðu; r; LÞ þ a3 ðu; l; RÞ þ a4 ðu; r; RÞ such that for all r ar is non negative, a1 þ a2 ¼ a3 þ a4 ¼ 1=2, and z2 ¼ ðd; l; LÞ, – g1 ¼ ðg1a ; g1b Þ ¼ ð0; 0Þ.
– – – –
Moreover, all equilibrium joint plans here are of this kind. 7. Larger spaces of states Concerning the existence of equilibrium for at least three states of nature, we only have the following negative result. Proposition 7.1. If jKj b 3, completely revealing equilibria and equilibrium joint plans may simultaneously fail to exist. Proof: Consider the following example, with K ¼ fa; b; cg and l r
p0 ¼ ð1=3; 1=3; 1=3Þ: l r
u
ð0; 2; 0Þ
ð0; 2; 0Þ
u
ð0; 0; 1Þ
ð0; 0; 1Þ
d
ð1; 1; 0Þ
ð1; 1; 0Þ
d
ð1; 0; 1Þ
ð1; 0; 1Þ
L
state a
l
r
u
ð1; 0; 3Þ
ð1; 0; 3Þ
d
ð0; 0; 3Þ
ð0; 1; 3Þ L
l
r
u
ð1=2; 1=2; 0Þ
ð1=2; 1=2; 0Þ
d
ð1; 0; 0Þ
ð1; 1; 0Þ
state b
l
r
u
ð1; 1; 0Þ
ð1=2; 0; 0Þ
d
ð1; 1; 0Þ
ð3=2; 0; 0Þ L
R
R l
r
u
ð0; 1; 1Þ
ð0; 0; 1Þ
d
ð1; 1; 1Þ
ð0; 0; 1Þ
state c
R
In comparison with the previous example (see the last part of the proof of theorem 6.1), we have only changed the payo¤s in state b if the uninformed player plays R and added a new state c. As usual, vik ¼ 0 for any informed player i and state k. For all p, u3 ðpÞ ¼ vex u3 ðpÞ ¼ maxf3p b ; p a þ p c g. Note that for any z in DðSÞ, g1b ðzÞ þ 3=4g2a ðzÞ b 1=2 and g1c ðzÞ þ g2b ðzÞ b 1=2. Finally, the stars denote, for each state, the extreme points of the Nash equilibria of the game in this state: for example, in G a the set of equilibria is flðu; l; RÞ þ ð1 lÞðu; r; RÞ : l A ½0; 1g. First, no CR equilibrium can exist, since the payo¤s for the informed
3-player repeated games with lack of information on one side
241
players would be 0 in each state and the equilibrium payo¤ would not belong to JR1; 2 ðb; aÞ and JR1; 2 ðc; bÞ. An intuition of the problem (due to the fact that there are more than 2 states) of finding an equilibrium joint plan here is the following. There will be some message after which player 3 will essentially play L, and this case will happen when the a posteriori of player 3 assigns positive probability to state b. Consequently in this case player 1 will play d and player 2 will play l for individual rationality reasons. So we are left with a vector payo¤ for the informed players which is: 1 0 1 0 a j1 j2a 1 1 C B C B b 0 A: @ j1 j2b A ¼ @ 0 j1c j2c 1 1 Since it contains negative numbers, ðd; l; LÞ can not be played at every stage in any state. If we consider an EJP for player 2, it must give a payo¤ not lower than 1 for player 2 in state a because of the incentive condition. So when state a is selected, player 3 has to play L at least half of the time. This will be di‰cult because of the IR condition for player 3, who need to play R as soon as his a posteriori p satisfies p b < 1=4. Similarly, if we consider an EJP for player 1, it must give a payo¤ not lower than 1 for player 1 in state c. So when this state is selected, player 3 has to play L su‰ciently many times whereas player 2 has to play r. Again, this will be di‰cult because of the IR condition for player 3. We now formally conclude the proof. Assume that there exists an equilibrium joint plan ðM; l; P; z; gi Þ for some informed player i. We are going to find a contradiction. First notice that for pm in P, pma > 0 implies that player 1, by individual rationality, plays u within zm . Similarly, pmc > 0 implies that player 2 plays r within zm . P Since p0 ¼ m A M lm pm , let p1 , p2 and p3 be in P such that p1a b 1=3, p2b b 1=3 and p3b b 1=3. Since 3 p2b > p2a þ p2c , by individual rationality player 3 must play L within z2 . This induces z2 ¼ ðd; l; LÞ hence: 0 a 1 0 1 g1 ðz2 Þ g2a ðz2 Þ 1 1 B b C B C 0 A: @ g1 ðz2 Þ g2b ðz2 Þ A ¼ @ 0 g1c ðz2 Þ g2c ðz2 Þ 1 1 By individual rationality of the informed players, we must get p2 ¼ db . Using lemma 5.3, for any m in M one must have: pma > 0 implies g2a ðzm Þ b 2=3; pmc > 0 implies g1c ðzm Þ b 1=2: About message 1, since p1a > 0 player 1 must play u within z1 . Moreover, we have just seen that g2a ðz1 Þ should be at least 2/3. What can be the support of p1 ? A priori, it belongs to ffag; fa; bg; fa; cg; fa; b; cgg. But fag and fa; cg are impossible, since in this case by individual rationality player 3 should play R within z1 . fa; b; cg is also impossible, since player 2 should play r because
242
J. Renault
of state c and player 3 should play L in order to have g1c ðz1 Þ b 1=2: but in this case g1b ðz1 Þ < 0, contradicting point i) of definition 5.2. Hence we are left a1 ðu; l; LÞ þ a2 ðu; r; LÞ þ with a support of fa; bg for p1 . Let z1 be denoted Pby 4 a3 ðu; l; RÞ þ a4 ðu; r; RÞ, with ar b 0 for all r and r¼1 ar ¼ 1. Put a ¼ a1 þ a2 . For player 1 to be individually rational in state b, we must have a þ 1=2 ð1 aÞ b 0 hence a a 1=3. But this gives g2b ðz1 Þ > 0, so using the incentive condition and since g2b ðz2 Þ ¼ 0, the joint plan can not be a joint plan for player 2. Thus, it is a joint plan for player 1, and g2a ðz1 Þ b 2=3 gives a ¼ 1=3. Hence the support of p1 is fa; bg, z1 ¼ a1 ðu; l; LÞ þ a2 ðu; r; LÞ þ a3 ðu; l; RÞ þ a4 ðu; r; RÞ with a1 þ a2 ¼ 1=3, and 0 a 1 0 1 g1 ðz1 Þ g2a ðz1 Þ 0 2=3 B b C B C 1=3 A: @ g1 ðz1 Þ g2b ðz1 Þ A A @ 0 g1c ðz1 Þ g2c ðz1 Þ a1=3 a0 Note also that the IR condition for the uninformed player gives ðp1a ; p1b ; p1c Þ ¼ ð3=4; 1=4; 0Þ. We now consider message 3. Since p3c > 0 player 2 must play r within z3 and g1c ðz3 Þ should be at least 1/2. Now, Supp p3 A ffcg; fa; cg; fb; cg; fa; b; cgg. fcg and fa; cg are impossible because they induce R for player 3, contradicting g1c ðz3 Þ b 1=2. Again, fa; b; cg is not possible since it induces u for player 1, and then L for player 3 giving g1b ðz3 Þ < 0. We are thus left with Supp p3 ¼ fb; cg. r; RÞ þ b 4 ðd; r; RÞ, with Let z3 be denoted by P b4 1 ðu; r; LÞ þ b 2 ðd; r; LÞ þ b3 ðu; b r b 0 for all r and r¼1 br ¼ 1. We must have g1b ðz3 Þ ¼ 0 by the incentive condition, thus b 1 þ 1=2b3 þ b4 ¼ 0. Eliminating b1 and b 2 , we obtain: 0 a 1 1 0 g1 ðz3 Þ g2a ðz3 Þ 1 þ 3=2b3 þ b 4 1 1=2b3 B b C C B 0 2b 3 þ 3b4 1 A: @ g1 ðz3 Þ g2b ðz3 Þ A ¼ @ g1c ðz3 Þ g2c ðz3 Þ 0 3=2 2b3 5=2b4 Individual rationality for player 2 in state b yields 2b3 þ 3b 4 1 b 0. But then, 3=2 2b 3 5=2b4 ¼ ð1 2b3 3b 4 Þ þ ð1=2 þ 1=2b4 Þ a 1=2 þ 1=2b4 . Hence 3=2 2b 3 5=2b4 < 1, so g1c ðz3 Þ < g1c ðz2 Þ. The incentive condition for the first player can not be satisfied. Thus no equilibrium joint plan exists for this game. This concludes the proof of proposition 7.1. rrr The last proposition shows that the consideration of CR equilibria and equilibrium joint plans is not su‰cient. We then have to consider more general classes of equilibria that might always exist for this 3-person case of lack of information on one side. A first idea could be to consider analogues of joint plans where both informed players would simultaneously select some message. However, we do not think this would easily solve our problem, partly because of the di‰culty of choosing the order of the announcements of the selected messages. A more accurate idea, in our opinion, would be to consider successions of joint plans: first, player 1 could send a message depending on the state, then player 2 could send another message depending on the state and on the first message. These two messages (or more, if player 1 is allowed to send a third message depending on the state and on the first two messages, etc.) would, as in original joint plans, induce a path of pure actions . . .
3-player repeated games with lack of information on one side
243
A third and simpler idea is motivated by the previous example. Consider the following joint plan for player 1: – M ¼ f1; 2; 3g. – l a ¼ ðl1a ; l2a ; l3a Þ ¼ ð1; 0; 0Þ, l b ¼ ð1=3; 1=3; 1=3Þ, l c ¼ ð0; 0; 1Þ. – p1 ¼ ðp1a ; p1b ; p1c Þ ¼ ð3=4; 1=4; 0Þ, z1 ¼ 1=3ðu; l; LÞ þ 2=3ðu; l; RÞ, with payo¤ 0
1 0 g1a ðz1 Þ g2a ðz1 Þ 0 B b C B b @ g1 ðz1 Þ g2 ðz1 Þ A ¼ @ 0 g1c ðz1 Þ g2c ðz1 Þ 1=3
1 2=3 C 1=3 A: 1
– p2 ¼ ð0; 1; 0Þ, z2 ¼ ðd; l; LÞ, with payo¤ 0
1 0 g1a ðz2 Þ g2a ðz2 Þ 1 B b C B @ g1 ðz2 Þ g2b ðz2 Þ A ¼ @ 0 g1c ðz2 Þ g2c ðz2 Þ 1
1 1 C 0 A: 1
– p3 ¼ ð0; 1=4; 3=4Þ, z3 ¼ 2=5ðu; r; LÞ þ 1=5ðd; r; LÞ þ 2=5ðd; r; RÞ, with payo¤ 0 1 0 1 g1a ðz3 Þ g2a ðz3 Þ 3=5 1 B b C B C 1=5 A: @ g1 ðz3 Þ g2b ðz3 Þ A ¼ @ 0 g1c ðz3 Þ g2c ðz3 Þ 1=2 0 Recall what is the underlying strategy of this joint plan. The informed players observe the true state k, then player 1 chooses m in M according to l k and announces m to the other players. Finally, the three players play an infinite sequence of pure actions leading to zm . Concerning punishments, if one of the informed players deviates from the path leading to zm (or if player 1’s announcement is out of any code) he is punished to his minmax in G k , whereas if the uninformed player deviates from the path to zm he is punished to vex u3 ð pm Þ. This is not an equilibrium here, because when the true state is c player 1 can deviate by announcing the message 2 and have a payo¤ of 1 instead of 1/2. But a slight modification of this strategy can lead to a uniform equilibrium. Just after the announcement of the message, add a communication phase only in case the message announced is 2. Player 2 is then asked to tell, using codes on actions, one of the three following a‰rmations according to the truth: ‘‘OK, I agree with the announce of player 1’’, or ‘‘No, the true state is a’’, or ‘‘No, the true state is c’’. If player two says one of the first two a‰rmations, the play goes on as before, with an infinite sequence leading to z2 . But if player 2 announces the true state is c, then the three players play ðu; r; LÞ at any stage. Punishments are similar as before. In any case, if some informed player j deviates from the path of pure actions (or makes an announcement ‘‘out of any code’’), the other informed player has to announce (if it is not already done) the state of nature k, and then players in Nnf jg punish player j to his minmax in G k . If the uninformed player deviates from the path leading to zm , the informed players punish him to vex u3 ðpm Þ. Let us see why this is a uniform equilibrium. It is plain that player 3 has
244
J. Renault
no profitable deviation. Considering deviations of an informed player, the key point is that when player 1 announces the message 2 and player 2 replies ‘‘No, the true state is c’’, player 3 does not know who is deviating but ðu; r; LÞ is enough to prevent all possible deviations. If player 1 deviates by announcing message 2 whereas the state is c, he obtains with ðu; r; LÞ a payo¤ of 1/2 in state c, and this is not greater than what he gets without deviating. Finally, if player 2 deviates by replying ‘‘No, the true state is c’’ whereas it is b, he obtains with ðu; r; LÞ a payo¤ of 0 which is not greater than what he gets without deviating. Back to the general case, an interesting idea is thus to define generalizations of joint plans where after the announcement of the message m by one of the informed player, say i, the other informed player (3 i) has to say ‘‘OK’’ or Then ‘‘No, the true state is k such that lmk ¼ 0, hence player i has deviated’’. 0 player i would be asked to reply ‘‘No, the true state is k 0 and lmk > 0. Player 3 i did deviate by accusing me’’. In this case the play followed afterwards should be such that both informed players should be punished if they actually deviated. This new kind of joint plan could be called a ‘‘controlled joint plan’’ and generalizes both the ideas of completely revealing equilibrium and equilibrium joint plan. Using this relatively simple notion, it may be possible to prove the existence of a uniform equilibrium for any number of states. References [Au-Ma]
Aumann RJ, Maschler M (1995) Repeated Games with Incomplete Information. With the collaboration of R. Stearns. Cambridge, MA: MIT Press [Ha] Hart S (1985) Nonzero-sum two-person repeated games with incomplete information, Mathematics of Operations Research, 10:117–153 [Si-Sp-To] Simon RS, Spiez˙ S, Torun´czyk H (1995) The existence of equilibria in certain games, separation for families of convex functions and a theorem of Borsuk-Ulam type, Israel Journal of Mathematics 92:1–21 [So] Sorin S (1983) Some Results on the Existence of Nash Equilibria for Non-Zero Sum Games with Incomplete Information, International Journal of Game Theory, 12:193–205 [To] Tomala T (1999) Nash Equilibria of Repeated Games with Observable Payo¤ Vectors, Games and Economic Behavior, vol. 28, 2:310–324
8. Appendix The proof of (1) is due to the fact that ð pt Þtb0 is a ðHt Þtb0 -martingale with values in DðKÞ and with expectation p0 : Ek A K; Et b 0;
k k Eðð ptþ1 ptk Þ 2 Þ ¼ EðEðð ptþ1 ptk Þ 2 j Ht ÞÞ k Þ 2 Þ Eðð ptk Þ 2 Þ: ¼ Eðð ptþ1
P T1
2
2
k ðptþ1 ptk Þ 2 Þ ¼ Eðð pTk Þ Þ ðp0k Þ a p0k ð1 p0k Þ. (1) now fol-ffi rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 P T1 k 1 P T1 k k Eð t¼0 ð ptþ1 ptk Þ 2 Þ lows from the fact that E t¼0 jptþ1 pt j a T T for any k.
Hence Eð
t¼0
We now prove (2). For simplicity, we will sometimes omit to mention the
3-player repeated games with lack of information on one side
245
action played by player 3 at the last stage when writing a posteriori. Note that for any t b 0 and ht in Ht , we have for any pair of actions ðs1 ; s2 Þ in S1 S2 played by the informed players at stage t þ 1, and any state k: k ðht ; s1 ; s2 Þ ¼ Pðk j ht ; s1 ; s2 Þ ptþ1
¼
Pðkjht ÞPðs1 ; s2 j k; ht Þ Pðs1 ; s2 j ht Þ
ptk ðht Þ s1k ðht Þðs1 Þ s2k ðht Þðs2 Þ : sðht Þðs1 ; s2 Þ P P k Since Eðkptþ1 pt k j ht Þ ¼ k A K ðs1 ; s2 Þ A S1 S2 sðht Þðs1 ; s2 Þj ptþ1 ðht ; s1 ; s2 Þ k pt ðht Þj, we obtain: ¼
Eðk ptþ1 pt k j ht Þ X X ¼
ptk ðht Þjs1k ðht Þðs1 Þ s2k ðht Þðs2 Þsðht Þðs1 ; s2 Þj
k A K ðs1 ; s2 Þ A S1 S2
¼
X
ptk ðht Þks1k ðht Þ n s2k ðht Þ sðht Þk
kAK ~
~
¼ Eðks1k ðht Þ n s2k ðht Þ sðht Þk j ht Þ:
