MATHEMATICS OF OPERATIONS RESEARCH Vol. 27, No. 2, May 2002, pp. 419–440 Printed in U.S.A.

REPEATED GAMES WITH LACK OF INFORMATION ON ONE SIDE: THE DUAL DIFFERENTIAL APPROACH RIDA LARAKI We introduce the dual differential game of a repeated game with lack of information on one side as the natural continuous time version of the dual game introduced by De Meyer (1996). A traditional way to study the value of differential games is through discrete time approximations. Here, we follow the opposite approach: We identify the limit value of a repeated game in discrete time as the value of a differential game. Namely, we use the recursive structure for the finitely repeated version of the dual game to construct a differential game for which the upper values of the uniform discretization satisfy precisely the same property. The value of the dual differential game exists and is the unique viscosity solution of a firstorder derivative equation (which appears implicitely in De Meyer and Rosenberg 1999) with a limit condition. We identify the solution by translating viscosity properties in the primal.

Introduction. Repeated games with incomplete information on one side were first introduced and studied in 1966–1968 by Aumann and Maschler (1995): At each stage, two players play a zero-sum I × J matrix game where the matrix of payoffs Ak = Aki j i∈I j∈J depends on a parameter k which is selected in a finite set K before the play according to some probability p in K (the set of probabilities over K) and announced to Player 1 only (see §1 for a formal definition). Aumann and Maschler (1995) proved that the value of the n-stage repeated game vn p

(resp. the value of the -discounted game v p ) is a concave function on p, and that the family converges uniformly as n →  (resp. as  → 0) to Cav K u p . Here, up is  the value of the average game Gp

(in which neither player is told anything about the choice of chance (or equivalently the game where the informed player does not use his information)) and Cav K u is the smallest function, concave and greater than u on K . Moreover, they prove that the error term n p = vn p − Cav K u p

√ (resp.  p = √ v p −Cav K u p ) is uniformly bounded by A / n (resp. A ). (A denotes the maximum absolute value of the one stage payoff.) Here, we give an alternative proof of this result and some new interpretations by considering the dual approach, following previous work of De Meyer (1996) and De Meyer and Rosenberg (1999). The dual game was introduced by De Meyer (1996) to study the error term. He computed the Fenchel-Legendre conjugate wn of vn and proved that wn corresponds to the value of what he called “the dual game.” More recently, De Meyer and Rosenberg (1999) used an operator approach in the dual to prove Aumann and Maschler’s (1995) Theorem. They showed that the limit, h, of wn is approximately a fixed point of a family of operators Tn . If h were regular C 2 , then a Taylor expansion would give for n large:
  1 1 1 Tn h − h z = max min − hz + z − Ai y hz + O 2  i∈I y∈ J

n+1 n+1 n Thus, a first order differential equation in h appears. Received July 14, 1999; revised June 13, 2000, and March 21, 2001. MSC 2000 subject classification. Primary: 91A20, 91A23. OR/MS subject classification. Primary: Games/group decisions, differential. Key words. Differential game, repeated game. 419 0364-765X/02/2702/0419/$05.00 1526-5471 electronic ISSN, © 2002, INFORMS

420

R. LARAKI

Actually, assuming that nTn h − h → 0, h should satisfy: DMR

max min−hz + z − Ai y hz  = 0

y∈ J i∈I

The first motivation of this paper was to give a formal justification for the heuristic apparition of DMR . Actually, we prove that wn is also the value of a uniform discretization of a differential game between 0 and 1 (that we call the dual differential game). We deduce that lim wn exists and satisfies DMR , which appears as the Hamilton-Jacobi-Isaacs equation of the differential game, in the viscosity sense (see Appendix, Definition A). It is the unique solution, hence completely characterized, if a limit condition is added to DMR

(see equation H , §4). The idea of studying the asymptotic value of a discrete time repeated game as the value of a game in continuous time (with fixed duration) appears in Sorin (1984) in the case of Big Match with incomplete information on one side. Vieille (1992) was the first to introduce a differential game (with fixed duration) to study weak approachability for repeated games with vector payoffs (Blackwell 1956). Our dynamics is the same as the one introduced by Vieille (1992) (see §3). Basically, one way to study the value of differential games is through approximation by sequence of discretizations (see, e.g., Fleming 1961, 1964, Barron et al. 1984, Souganidis 1985a, 1985b). We use the opposite approach here: We construct a differential game for which wn is the value of an uniform discretization. In §1.1, we recall the description of a repeated game with incomplete information on one side and some basic properties. Section 1.2 is devoted to the dual game and FenchelLegendre duality relations between the primal and the dual. In §2, using the dual recursive formula for wn n∈N , we introduce heuristically the dual differential game as a continuous time version of the dual game. Then, we prove rigorously the fact that wn (resp. w ) corresponds to the value of a discretization of the dual differential game. In §3, results from the theory of differential games allow us to show that the values of the discretizations of the dual differential game converge to the viscosity solution of the Hamilton-Jacobi-Isaacs equation with a boundary condition E associated to the dual differential game. Since Fenchel-Legendre conjugacy is an isometry on the space of proper (i.e., not identically +) convex lower semicontinuous functions from
K →
∪ #+$ (see Rockafellar 1970), we obtain as a corollary the convergence part of Aumann and Maschler’s (1995) Theorem. In §4, we give a rigorous justification to DMR : Namely, we prove that DMR derives from the Hamilton-Jacobi-Isaacs equation E by using a time homogeneity property. In fact, the asymptotic value is completely characterized by Equation H , obtained by adding to DMR a limit condition (which derives from the boundary condition of Equation E ). We study H in a general framework (not related to game theory) and prove an existence and uniqueness result. We could identify the asymptotic value of the dual (as the Fenchel-Legendre conjugate of u) using exclusively viscosity solution theory by applying Hopf’s formula (Bardi and Evans 1984, see Theorem F in the Appendix) since the boundary condition of the Hamilton-JacobiIsaacs equation E is convex. We could also verify directly that the Fenchel-Legendre conjugate of u is a viscosity solution. Here we follow a constructive approach which consists in translating the problem of identification from the dual to the primal: We use the characterization in the dual (namely the fact that we have a viscosity solution) to derive necessary and sufficient conditions on v = lim vn .

REPEATED GAMES WITH LACK OF INFORMATION

421

Hence, in §5, we prove that v is the unique concave continuous function on K

satisfying the following two properties: • P1 for all p0 , extreme point (of the hypograph) of v : v p0 ≤ up0 , • P2, v ≥ u, from which we deduce that v = Cav K u . Actually these conditions extend naturally to games with incomplete information on both sides and lead to alternative formulations (Laraki 2001). In §6, we extend Aumann and Maschler’s (1995) result to generalized payoffs. Finally, the Appendix recalls results concerning viscosity solutions that will be used. 1. Zero sum repeated games with lack of information on one side. This section is devoted to the presentation of the repeated game with lack of information on one side, called here the primal game. 1.1. The primal game. This model has been introduced in 1966 by Aumann and Maschler (1995) and has been extensively studied since (see, for example, Mertens et al. 1994, Chapter 5). 1.1.1. Definitions. Let I J  K be finite sets, Ak = Aki j

k∈K be a family of I × J matrices, and A = maxk i j Aki j . There are two players: Player 1 (the maximizer) and Player 2 (the minimizer). For each p in K (the set of probabilities over K), the game form Gp is described as follows: • At stage 0, k is chosen according to p and announced to Player 1 only. • At stage 1, Player 1 chooses a move i1 ∈ I, Player 2 chooses a move j1 ∈ J . Then, the couple i1  j1 is told to both. The stage payoff is g1 = Aki1  j1 but it is not announced. • Inductively, at stage m, knowing the past history hm = i1  j1  ' ' '  im−1  jm−1 , Player 1 chooses a move im ∈ I, Player 2 chooses a move jm ∈ J . Then, the new history hm+1 = hm  im  jm is told to both. The stage payoff is gm = Akim  jm but again is not announced. • Both players know the above description (public knowledge).  Let Hm = I × J m−1 be the set of histories at stage m (H1 = #$) and let H = m≥1 Hm be the set of all histories. We denote by X = I and Y = I the set of mixed moves of the players. A (behavioral) strategy for Player 1 is a map * from K × H to X. Similarly, a strategy for Player 2 is a map + from H to Y . We denote by , (resp.  ) the set of strategies of Player 1 (resp. Player 2). A triple p * + in K × , ×  induces a probability distribution Pp * + on the set K × I × J  of plays (endowed with the *-field generated by the cylinders). Ep * + stands for the corresponding expectation. Several games are associated to this game form; they differ only in the way the stream of payoffs #gm = Akim  jm $ is evaluated. First, we will be interested in the finitely repeated game Gn p and the -discounted game G p (0 <  < 1): n • Gn p is n-stage game with payoff Ep * + 1/n

m=1 gm .  • G p is the game with payoff Ep * +  m=1 1 −  m−1 gm . The stage payoffs being uniformly bounded, when we endow , and  with the product topology, these sets are compact and the payoff functions are jointly continuous and bilinear on , ×  . Hence, Gn p and G p have a value vn p and v p , respectively, according to the minmax theorem (Sion 1958). We denote by up , for p ∈ K , the value of the one shot zero-sum (average) game   Gp

with I × J (average) matrix payoff: Ap = k∈K pk Ak ; this is also the value of the game in which Player 1 is restricted to nonrevealing strategies (or in which he has no information). A nonrevealing strategy for Player 1 is a strategy where he plays independently on his private information (as if he has no information).

422

R. LARAKI

1.1.2. General properties. In this section, we recall some very well-known properties that the author can find, for example, in Mertens et al. (1994). Proposition 1. vn p and v p are concave and Lipschitz with constant A and satisfy the following formulas:   k k k  nvn p = max min p x A y + n − 1 xi v ¯ Rn

n−1 pi

 x∈X K y∈Y

 R

v p = max min  x∈X K y∈Y

i∈I

k∈K

 k∈K

k k

k

p x A y + 1 − 

 i∈I

xi v ¯  pi



with  ¯ = k∈K pk xk i . • xi

¯ if xi

¯ = 0 and pk i = pk if xi

¯ = 0. Namely: pi is the • pk i = pk xk i /xi

conditional distribution on K given i, induced by p and x ∈ XK .  k k k k A y = x i A yj . x • i∈I j∈J i j 1.1.3. Aumann and Maschler’s (1995) theorem. The main purpose of this paper is to give an alternative proof and a new interpretation of the famous result from Aumann and Maschler (1995). Recall that Cav K u denotes the smallest function, concave and greater than u on K . Theorem 1. vn (resp. √v ) converges √ uniformly as n →  (resp. as  → 0) to Cav K u with speed 1/ n (resp. ). Explicitly, there exists √ a constant C, such that √ vn − Cav K u  ≤ C/ n (resp. v − Cav K u  ≤ C ). The original proof is in two parts. The lower bound comes from the concavification operator as the expression of the use of information. The upper bound is a maxmin argument using the convergence property of the martingale of beliefs of Player 2 (in K ) generated by a strategy of the informed Player 1. The proof of De Meyer and Rosenberg (1999) uses an operator approach on the Fenchel conjugate wn of vn (see next section). Here, we also start from the dual but use differential game theory to prove the convergence and to characterize the limit. 1.2. The dual game. The fact that the functions vn and v are concave leads De Meyer (1996) to compute their Fenchel-Legendre conjugates. He then proved that they correspond to the values of a game which he called the dual game. In fact the dual game can be defined for a large class of games with incomplete information (Sorin 2002), independently of the repeated aspect. This is the game where the informed player (Player 1) is allowed to choose the state k ∈ K at the beginning of the game, but where we add a cost (a control variable), which depends on the state chosen. 1.2.1. Definitions and properties. We follow Sorin (2002). Definition 1: Game with Incomplete Information. Let S and T be two convex subsets of a topological vector space and K be a finite set. Let k k∈K be a family of bilinear and uniformly bounded functions from S × T to
. The game with incomplete information p is a two-stage, zero-sum game, where: • At stage 0, the state k ∈ K is drawn according to p ∈ K , then Player 1 is informed but not Player 2. • At stage 1, the players choose a move in their strategy set, s ∈ S and t ∈ T . • The payoff is k s t .

REPEATED GAMES WITH LACK OF INFORMATION

423

A strategy of Player 1 (resp. Player 2) is an element s ∈ S K (resp. t ∈ T ). The associated payoff is:  k k k p s t = p  s  t  k∈K

Notations vp = supS K inf T p s t (the maxmin). vp

¯ = inf T supS K p s t (the minmax). Proposition 2 (Sorin 2002). v¯ and v are Lipschitz with constant C and concave They are equal (the value exists) if S (or T ) is bounded and if the k k∈K are separately continuous. Definition 2: The Dual Game of a Game with Incomplete Information. For each vector z ∈
K , we define the dual game of p  denoted by  ∗ z , as follows: • At stage 0, Player 1 chooses k. • At stage 1, both players choose a move s ∈ S and t ∈ T . • The payoff is k s t − zk . A strategy for Player 1 (resp. Player 2) in  ∗ z is an element p s ∈ K × S K (resp. t ∈ T ). The associated payoff is: ∗z s t =



pk k s k  t − zk 

k∈K

Notations ¯ and wz denote the maxmin and the minmax of  ∗ z . • wz

¯ • 1 is the vector on
K with all components equal to 1. ˜ (resp. wp ) ˜ stands for vp

¯ or vp (resp. wp

¯ or wp ). • vp

Proposition 3 (Sorin 2002). w˜ is convex and Lipschitz with constant 1, satisfies ¯ = wz

¯ and the Fenchel-Legendre duality relations hold: wz ˜ + a1

˜ − a1, wz

˜ = max #vp

˜ − p z $ p∈ K

˜ + p z $ vp

˜ = inf #wz

z∈
K

In particular, if the primal game has a value, then the dual game has a value and reciprocally. Remark 1. De Meyer (1996) uses the usual convention of the Fenchel-Legendre conjugate of a function v: hz = minp∈ K #p z − vp $ = −wn z . In his approach, Player 1 minimizes the payoff and Player 2 maximizes it. We follow here Sorin’s (2002) convention in order to be consistent with the habit of considering Player 1 as the maximizer in zero-sum games. 1.2.2. Application to finitely repeated and discounted games. As seen in §1, the finitely repeated game Gn p and the -discounted game G p can be normalized as a one-stage game with compact strategy spaces and with incomplete information on one side: Nature chooses the state k ∈ K according to the probability p ∈ K , Player 1 is informed and Player 2 is not, then the players choose their moves in their set of strategies (, and  ).

424

R. LARAKI

Thus, as above, one can define their dual games G∗n z and G∗ z and we have the following: Proposition 4 (De Meyer 1996). G∗n z (resp. G∗ z ) has a value, wn z (resp. w z ) which satisfies all the properties of Proposition 3. Moreover, wn z and w z satisfy the recursive formulas:      n n+1 1   w z − Ai y  wn+1 z = min max

∗ y∈Y i∈I n+1 n n n Rn  w0 z = max−zk  k

 1  z− Ai y  y∈Y i∈I 1− 1−  with Ai y = A1i y ' ' '  Aki y ' ' '  AKi y t and Aki y = j∈J Aki j yj .



R∗





w z = min max1 −  w

Remark 2. The dual game also presents an interest for the strategies. Actually, there is a correspondence between primal and dual optimal strategies. This property allows the construction of optimal strategies for the uninformed player in the primal game (see Rosenberg 1998). 2. The dual differential game. The recursive formula Rn∗ in the dual game will enable the construction of a differential game for which wn appears as the value of a discretization. 2.1. Motivation-definition. In order to obtain a differential game having a value, let us transform slightly Rn∗ . Because of the convexity of the functions wn n + 1 /n z − 1/n xAy

n∈N in x (as a composition of a convex and of a linear function) the maximum in x is achieved at some extreme point (i ∈ I) of the simplex X. Thus the recursive formula Rn∗ can be rewritten as:      n n+1 1   z

= min max w z − xAy  w

∗  n+1 y∈Y x∈X n+1 n n n Rn  w0 z = max−zk  k

with xAy = xAk y k∈K . Suppose that the players play in a zero-sum differential game between 0 and 1. The initial state variable is z0 = z ∈
K and Player 1 (resp. Player 2) chooses at each time t ∈ 0 1, a move xt ∈ X (resp. yt ∈ Y . Assume that the moves induce the following dynamics on the state space dzk t = −xt Ak yt dt. Finally, at t = 1, Player 1 receives the payoff gz1  = maxk −zk 1

. The nth uniform discretization of this differential game is the n-stage repeated game where players move only at discrete time m/n 1≤m≤n and where the state variable moves according to the discrete dynamics zm/n − zm − 1 /n = −1/n xm Aym . Its upper value (see §2.2) is Un+ :   n 1 + Un z = inf sup · · · inf sup g z − x Ay  x1 ∈X y ∈Y xn ∈X y ∈Y n m=1 m m 1 n Now, since g is positively homogeneous, we deduce that     n  1 n+1 n n+1 1 1 g z−  g z − x1 Ay1 − x Ay = x Ay n + 1 m=1 m m n+1 n n n m=1 m+1 m+1

REPEATED GAMES WITH LACK OF INFORMATION

Thus,

 + Un+1 z = inf sup y∈Y x∈X

425

   1 n n+1 Un+ z − xAy  n+1 n n

so that the recursive formula Rn∗ holds for Un+ n∈N . Therefore, we are led to the following definition: Definition 3: The Dual Differential Game. The dual differential game with initial state z ∈
K , 6 z , is a zero-sum differential game with duration 1, where • The state space is Z =
K . • T = 0 1 is the time interval of the game. • X = I and Y = J . • Player 1 uses a measurable control x 0 1 → X. • Player 2 uses a measurable control y 0 1 → Y . • If Player 1 uses x and Player 2 uses y, then the dynamics of the system is given by dzk t

= −xt Ak yt  dt

z0 = z

• The current payoff is zero. • The final payoff (at t = 1) from Player 2 to Player 1 is gz1  = maxk −zk 1

. Remark. The dynamics are the same as the ones used by Vieille (1992) to study weak approachability. Actually, Vieille (1992) proves that weak approachability reduces to the study of the following n-stage repeated zero-sum game: At each stage m m = 1 ' ' '  n, knowing the past history of moves, Player 1 chooses a move im ∈ I and Player 2 a move  jm ∈ J and the goal of Player 2 is to minimize the distance between nm=1 1/n Aim  jm ∈
K n K and C ⊂
 distC  m=1 1/n Aim  jm . 2.2. The Fleming discretization of the dual differential game. There are many different approaches to define upper and lower values for a differential game. When some regularity conditions are satisfied by payoff and transition functions, all the usual concepts coincide, and the upper (resp. the lower) value is characterized as the viscosity solution of the upper (resp. the lower) Hamilton-Jacobi-Isaacs equation with a boundary condition associated to the differential game (see, e.g., Fleming 1961, 1964, Barron et al. 1984, Souganidis 1985a, b). When the upper and the lower equations are the same (the so-called Isaacs condition is satisfied) the differential game has a value. For our model, most of the regularity conditions needed to obtain existence of the value are satisfied Since we will study repeated games by discretizing differential games of fixed duration, we will only present the concept of value introduced by Fleming (1961, 1964). Souganidis (1985b) generalized this approach by considering any finite subdivision and gave the rate of convergence in the “bounded Lipschitz final payoff case.” Here, we adapt this model to our situation with countable subdivisions and Lipschitz final payoff. Before, let us introduce some notations concerning subdivisions: • 8 = tm 0≤m≤  0 = t0 ≤ t1 ≤ · · · ≤ tm ≤ · · · ≤ t = 1, denotes any countable subdivision of the interval 0 1. The notation 8F stands for finite subdivision. • For each integer N ≥ 1 and each subdivision 8 = tm 0≤m≤  8 ∧ N denotes the finite subdivision 8 ∧ N = t0  t1  ' ' '  tm  ' ' '  tN −1  tN  1 . • :m = tm+1 − tm is the mth increment and 8 stands for the mesh of the subdivision 8: 8 = supm :m . • 8n denotes the uniform subdivision of 0 1 in n intervals: 8n = tmn 0≤m≤n , with tmn = m/n. • 8   ∈0 1 denotes the countable subdivision of 0 1 induced by a discount factor  = 1. : 8 = tm 0≤m≤ , with tm =  + · · · + 1 −  m−1  m ≥ 1 t0 = 0 and t

426

R. LARAKI

We associate to 6 z and 8 a family of games adapted to 8 and with different starting times. Let 68 z tm be the repeated game where at each time tl (l ≥  m) the players choose simultaneously a move xl ∈ X and yl ∈ Y . The final payoff is: gz − l≥m :l xl Ayl . The majorant game 68+ z tm is similar to 68 z tm except that Player 2 announces his move to Player 1 at each stage before Player 1 chooses his move. This game has a value W8+ z tm , with + W8+ z tm = lim W8∧m+N

z tm

N → 



= lim inf sup · · · inf sup g z − N →

ym

xm

m+N −1

ym+N x m+N

l=m

 :l xl Ayl



In the same way, we define the minorant game 68− z tm and its value: W8− z tm . The following proposition enumerates the usual properties satisfied by the values. Proposition 5. W8− z tm ≤ W8+ z tm . W8+ z tm = miny∈Y maxx∈X W8+ z − :m xAy tm+1 . W8− z tm = maxx∈X miny∈Y W8− z − :m xAy tm+1 . W8± z 1 = maxk −zk . Proof. If the players choose at stage m the move x y and the state is z, then the new state at tm+1 will be z − :m xAy. They can then select a move to guarantee the maxmin (resp. the minmax) in the new game: 68± z − :m xAy tm+1 .
2.3. The dual game is an upper discretization of the dual differential game. We will prove in the following lemma that the sequences wn and W=+n  ·  0 (resp. the functions w and W=+  ·  0 ) satisfy the same recursive formula and have the same initial value (resp. are the unique fixed point of the same operator), thus they coincide. In fact, this holds by the very construction of the dual differential game. Notations • Wn± z t = W8±n z t  (8n is the uniform subdivision of 0 1 in n intervals.) • W± z t = W8± z t . (8 is the subdivision of 0 1 induced by the discount factor .) Proposition 6. wn z = W8+n z 0 , w z = W8+ z 0 . Lemma 1. The Time Homogeneity Property. ± Wn± z m/n = 1 − m/n Wn−m z/1 − m/n  0  In 68n z m/n , the play xm+1  ym+1  ' ' '  xn  yn induces the payoff   n n 1  1  k k k g z− x A yl = max −z + x A yl k n l=m+1 l n l=m+1 l  n  n−m n k 1 k = x A yl max − z + k n n−m n − m l=m+1 l    n  m 1 1 k k = 1− max − z + x A yl k n 1 − m/n n − m l=m+1 l     m 1 1 n−m k k = 1− x A yl+m+1  max − z + k n 1 − m/n n − m l=0 l+m+1

Proof. 

So, playing in 68n z m/n is equivalent to playing in 1 − m/n 68n−m z/1 − m/n  0 .


427

REPEATED GAMES WITH LACK OF INFORMATION

Lemma 2. + Wn+1 z 0 = miny∈Y maxx∈X n/n + 1

Wn+ n + 1 /n z − 1/n xAy 0 , + W z 0 = miny∈Y maxx∈X 1 −  W+ 1/1 − 

z − /1 − 

xAy 0 . Proof. Because of the recursive structure of the discrete values of the differential game (Proposition 5), we have  + Wn+1 z 0

=

+ min max Wn+1 y∈Y x∈X

 1 1 z− xAy  n+1 n+1

By the time homogeneity property (the previous lemma), we obtain  + Wn+1

1 1 z− xAy n+1 n+1

thus, + Wn+1 z 0



   1 + z − 1/n + 1

xAy = 1− Wn 0  n+1 1 − 1/n + 1



  n 1 + n+1 = min max Wn z − xAy 0  y∈Y x∈X n + 1 n n

By the same argument, we have W+ z 0 = min max W+ z − xAy 

y∈Y x∈X   z − xAy = min max1 −  W+ 0 y∈Y x∈X 1−   1  + = min max1 −  W z− xAy 0  y∈Y x∈X 1− 1−




Lemma 3. W1+ z 0 = w1 z . Proof.

w1 z is the value of the one-stage dual game. Thus,  w1 z = min max max

y∈Y x∈X K p∈ K



k

k

k

k

p x A y − z

k∈K

= min max maxxAk y − zk  y∈Y

=

x∈X

k∈K

W1+ z 0 

Since miny∈Y maxx∈X maxk∈K xAk y − zk  is the upper value of the uniform discretization in one interval of the dual differential game.
For all 0 <  ≤ 1, define the operator T which associates to a real-valued function > on
K , the function T > on
K , defined by  T > z = min max1 −  > y∈Y

x∈X

 1  z− xAy  1− 1−

Denote, for a real-valued function > on
K , > = supz∈
K >z . Let Fg be the space of real-valued function > on
K satisfying:       ?> · − g·  ≤ ?A    ? 

∀ 0 < ? ≤ 1

428

R. LARAKI

It is clear that both W+ · 0 and w · are fixed points of T and are in Fg , since    N  + m−1 1 −  xm Aym  W z 0 = lim inf sup · · · inf sup g z − N →

y1

x1

yN

xN

m=1

w z = max v p − p z  p∈ K

and maxp∈ K v p  ≤ A . Lemma 4. For all 0 <  ≤ 1, T has a unique fixed point in Fg . Proof. First, let us verify that if > ∈ Fg , then T > ∈ Fg  Actually,     z z  − gz = min max ?1 −  > − xAy − gz

?T >

y∈Y x∈X ? 1 −  ? 1 − 
  z  = min max ?1 −  > − xAy − gz − ?xAy

y∈Y x∈X 1 −  ? 1 −   + gz − ?xAy − gz  Thus (since > ∈ Fg , we have       ?T > z − gz  ≤ ?1 −  A + ?A = ?A    ? Now, if >1 and >2 are two functions in Fg then (by the usual induction argument) for any integer n, one has Tn >1 − Tn >2  ≤ 1 −  n >1 − >2   However, by assumption we have >1 − >2  ≤ >1 − g + g − >2  ≤ 2A

take ? = 1 

Hence, if >1 and >2 are two fixed points, then >1 − >2  ≤ 21 −  n A for all n; this implies uniqueness.
Proof of Proposition 6. The sequences wn z and Wn+ z 0 satisfy the same recursive formula and have the same value for n = 1, thus they coincide. (In fact, we could simplify by proving the equality for initial values at 0: w0 z = W0+ z 0 . Actually, w0 is the conjugate of 0 on K and by construction W0+ z 0 = gz .) Both w z and W+ z 0 are in Fg and are fixed points of T , thus they coincide.
3. Convergence of the discrete values. 3.1. The value of the dual differential game and the associated Hamilton-JacobiIsaacs equation. Extensions and notations. • For a finite subdivision 8F , we can extend W8±F z · to all t ∈ 0 1 by using the recursive structure in the last proposition. Actually, we define by backward induction the functions W8±F z t  t ∈ 0 1:  + −zk  W8F z 1 = gz = max k 

W8+ z t = min max W8+ z − tm − t xAy tm for t ∈ tm−1  tm  F F y

x

REPEATED GAMES WITH LACK OF INFORMATION

and

429

 − −zk  W8F z 1 = gz = max k

 W8− z t = max min W8− z − tm − t xAy tm for t ∈ tm−1  tm . F F y

x

• For a general (countable) subdivision 8, we define W8± z t as the uniform limit of as N tends to infinity. • We extend the value of the average game to all
K by        k k  k k uq = max min x q A y = min max x q A y 

± W8∧N z t

x∈X y∈Y

y∈Y

k∈K

x∈X

k∈K

• BUC
K denotes the space of bounded uniformly continuous real-valued functions on
K . • U is the space of continuous real-valued functions f z t on
K × 0 1 uniformly continuous in z uniformly in t. Theorem 2. The functions W8+ z t and W8− z t converge, as 8 goes to 0, to the same limit W z t . Bz t = W z 1 − t is the unique viscosity solution in the space U of the HamiltonJacobi-Isaacs equation with a boundary condition:  CD   z t − u−z Dz t

= 0 in
K ×0 1, Ct E

 Dz 0 = max−zk

in
K  k

Moreover, W z 0 is  Lipschitz with constant 1 in z and the convergence of W8+ z 0 is uniform with a speed 8. Explicitly, there exists a constant C, such that  sup W8+ z 0 − W z 0  ≤ C 8 z∈
K

To prove this theorem, we adapt the problem in order to use the powerful results of Souganidis (1985a) concerning approximation schemes (see the Appendix, Theorem E). To do so we first introduce some notation. Let F − E and F + E , for 0 ≤ E ≤ 1, be the mappings from BUC
K → BUC
K

defined by F − E f z = sup inf f z − ExAy  x∈X y∈Y

+

F E f z = inf sup f z − ExAy  y∈Y x∈X

Then, one has: Lemma 5. F ± satisfy F1 ' ' '  F7 (see the Appendix, F conditions), with H = −u. Proof of the Lemma. It is straightforward to check that F1 ' ' '  F6 are satisfied. Here we verify F7 (with H = −u): for F − , for example. Given D ∈ Cb2 
K , we have:     −     F E Dz − Dz

Dz − ExAy − Dz

   − u Dz

 =  sup inf − sup inf − Dz  xAy   y∈Y y∈Y E E x∈X x∈X    Dz − ExAy − Dz

  ≤ sup  − − Dz  xAy  E x∈X y∈Y ≤E

D2 D  4

where D2 D = max1≤lm≤K supz∈
K C 2 D/Czl Czm z .


430

R. LARAKI

Proof of Theorem 2. Consider, for R > 0 the auxiliary equation:   CD z t − u− Dz t

= 0 in
K ×0 1, z Ct ER

 in
K  Dz 0 = gR z

where

 max−zk if − R ≤ max−zk ≤ R   k k    if max−zk ≥ R gR z = R k     −R if max−zk ≤ −R k

then • gR is a bounded 1-Lipschitz function. • For q ∈
K  −u−q = − maxx∈X miny∈Y −q xAy = minx∈X maxy∈Y q xAy is a continuous function (Lipschitz with constant A ). Then, using Theorem C (in the Appendix) with H q = −u−q , we conclude that ER

has a unique bounded uniformly continuous viscosity solution DR . Since gR is Lipschitz with constant 1, we deduce by Proposition D (in the Appendix) that DR z t is also Lipschitz in z with constant 1. Let 8F be a finite subdivision of the interval 0 1, 8F = 0 = t0  ' ' '  tN 8F = 1 . ± Define by induction the functions DR 8F by 

± DR 8F z 0 = gR z 

±  ± ± if t ∈ti−1  ti  DR 8F z t = F t − ti−1 DR 8F · ti−1 z

Thus, using Theorem E (in the that sup

Appendix), we deduce that there exists a constant CR such

z∈
K  t∈0 1

 ± D

  ≤ CR 8F 1/2 

R 8F z t − DR z t

Let 1 − 8F be the finite subdivision 0 = 1 − tN 8F  ' ' '  1 − t0 = 1 . It is quite easy to see that for any finite subdivision 8F and any z such that z ≤ R, ± W8±F z t = DR+A z 1 − t    1−8F ± ± z 1 − t and DR+A z t are both defined by the same induction forActually, W1−8 F   8F mula and the same function at t = 0 for z s.t. z ≤ R. Moreover, for all t ∈ 0 1 and all possible trajectories zt starting at z such that z ≤ R we have zt  ≤ R + A . (More precisely, we have zt  ≤ R + tA 

Thus, ∀ R ∃ CR , such that  ±  W z t − DR+A z 1 − t  ≤ CR 8F 1/2  sup 8F  z ≤R t∈0 1

We conclude from this property that: • W8+F z t and W8−F z t converge uniformly, on each compact P, as 8F  → 0 to the same limit, say W z t , and there exists a constant CP such that the error term is bounded by CP 8F 1/2  • W z t = DR+A z 1 − t  ∀ R ≥ z  ∀ t ∈ 0 1 This last property implies in particular that W z 1 − t is a viscosity solution of E

and that it is continuous and Lipschitz with constant 1 in z. Thus, W z 1 − t is uniformly continuous in z uniformly in t. But there exists only one viscosity solution of E with this property (see Theorem B in the Appendix).

REPEATED GAMES WITH LACK OF INFORMATION

431

For general (countable) subdivision, we use the fact that W8± z t is the uniform limit of when N →  and that 8 = 8 ∧ N  for N large. Now, we use results from §6, where we will prove that W8+ · 0 (resp. W · 0 ) is Fenchel-Legendre conjugate of some concave A -Lipschitz function on K  v8 (resp. v ). Fix z and let pz ∈ arg maxp v8 p − p z , then ± z t

W8∧N

W8+ z 0 − W z 0 = maxv8 p − p z  − maxv p − p z  p

p

= v8 pz − pz  z − maxv p − p z  p

≤ v8 pz − pz  z − v pz − pz  z ≤ v8 pz − v pz  Now, let z0 ∈ arg minz W z 0 − pz  z  (take an element of the surgradient of the concave function v at pz (see Rockafellar 1970, §§23 and 30), z0 ∈ C + v pz ). Then in the same way as above we prove that v8 pz − v pz ≤ W8+ z0  0 − W z0  0  Now, since v is Lipschitz with constant A , we deduce that z0  ≤ A . Hence,     sup W8+ z 0 − W z 0  = sup W8+ z 0 − W z 0  z

z z≤A

Choose C to be the constant CP associated to the compact, P = #z0 ∈
K  z0  ≤ A $




Remark 4. Here the Isaacs condition is satisfied, since maxx∈X miny∈Y q xAy = miny∈Y maxx∈X q xAy . Now, note that with the original recursive formula Rn∗ , one can also construct a differential game (by replacing X by I in our definition). However, this differential game (with dynamics dzt = −Ai y dt) will neither have a value nor the same dynamics as Vieille (1992). 3.2. The convergence part of Theorem 1. Let f z = W z 0 = Bz 1 be the value of the dual differential game with duration 1 starting at z at time 0. Then: Corollary 1. wn z (resp. w√  z ) converges √ uniformly to the value of the dual differential game, f z , with speed 1/ n (resp. ).


Proof.

We use the fact that wn z = Wn+ z 0 (resp. w z = W+ z 0 ) and Theorem 2.

Let v on K be the Fenchel-Legendre conjugate of f : v p = inf f z + p z  z

We will prove below that v = Cav K u .

√ √ Corollary 2. vn (resp. v ) converges uniformly to v with speed 1/ n (resp. ).

Proof. f , being convex and Lipschitz (as the uniform limit of wn ), we also have that f z = maxp∈ K v p − p z . Since Fenchel-Legendre duality is an isometry for the infinite norm on the space of proper lower semicontinuous convex functions (since the Legendre-Fenchel transformation is contracting and the biconjugate of a convex lower semicontinuous function is the function itself), we deduce the convergence part of the Theorem of Aumann and Maschler (1995).


432

R. LARAKI

4. A justification of (DMR). The fact that the payoff function g is positively homogeneous will imply a similar property for W . This property and equation E will give the first-order derivative equation DMR . Notations. f z = W z 0 = Bz 1 . This is the value of the dual differential game with duration 1 starting at z at time 0. Proposition 7. ∀ z ∈
K , ∀ t ∈ 0 1, W z t = 1 − t f z/1 − t

, or equivalently, Bz t = tf z/t . Proof. We conclude from the time homogeneity property (Lemma 1) by using the uniform convergence of Wn± z t to W z t , the density of the dyadic in 0 1, and the uniform continuity of all these functions.
At a differentiability point z t of W , we have      CW z z z z t = −f +  f Ct 1−t 1−t 1−t and

 z W z t = f

 z  1−t

so that, ∀ z, ∀ t ∈ 0 1: −

        CW z z z z − u−z W = 0 ⇒ f −  f − u −f = 0 Ct 1−t 1−t 1−t 1−t

In fact, we can prove a much more precise result: Theorem 3. The value of the dual differential game, f z  is a viscosity solution of the first-order derivative equation, with limit condition   Dz − z  Dz − u− Dz

= 0   H

z   lim ?D = max−zk = gz  + k∈K ?→0 ? It is the unique convex continuous function satisfying H . More precisely, a convex continuous function J ∗ is a viscosity solution of H if and only if Kz t = tJ ∗ z/t is the viscosity solution of E in U . Proof. The proof is in two parts. In Part 1, we prove that f is a viscosity solution of H , and in Part 2, we prove uniqueness in the space of convex continuous functions. Part 1. Recall (see the Appendix, Definition A) that we need to prove that for all z0 , − (1) p ∈ Cloc f z0 ⇒ f z0 − z0  p − u−p ≥ 0, + (2) p ∈ Cloc f z0 ⇒ f z0 − z0  p − u−p ≤ 0, (3) lim?→0+ ?f z0 /? = maxk∈K −zk0 . Now, since Bz t = tf z/t is the unique viscosity solution of E in U (Theorem 1), we have (see Definition A in the Appendix): − 1 r p ∈ Cloc Bz0  t0 ⇒ r − u−p ≥ 0, +  2 r p ∈ Cloc Bz0  t0 ⇒ r − u−p ≤ 0, 3 limt0 →0 Bz0  t0 = maxk∈K −zk0 . It is clear that 3 implies (3). Let us show that 1 implies (1) (the proof for (2 ) implies 2 is similar). − Let p ∈ Cloc f z0 and let r = f z0 − z0  p . Now, since f is a convex Lipschitz function − (as the uniform limit of the convex Lipschitz functions Wn+ · 0 = wn ), Cloc f z0 coincides

REPEATED GAMES WITH LACK OF INFORMATION

433

with the subdifferential of convex analysis (see Rockafellar 1970, §23, and Barles 1995), that is, ∀ z, ∀ t > 0:     z z ≥ f z0 + − z0  p  f t t thus Bz t ≥ tf z0 + z − tz0  p  which implies that Bz t − Bz0  1 − z − z0  p ≥ tf z0 − f z0 + z0 − tz0  p ≥ t − 1 f z0 − z0  p  ≥ t − 1 r − Hence, we deduce that p r ∈ Cloc Bz0  1 (see the Appendix) and by 1 , we obtain

f z0 − z0  p − u−p = r − u−p ≥ 0 Part 2: Uniqueness. To prove uniqueness, let us introduce some notations and establish some lemmas. Let J be a concave proper (i.e., not identically −) upper semicontinuous function from
K to
∪ #−$. Let S be the effective domain of J  S = #p ∈
K  vp = −$. Let J ∗ be the Fenchel-Legendre conjugate of J, J ∗ z = supJp − p z  p∈S

The following lemma says that the limit condition (i.e., the fact that J ∗ needs to satisfy lim?→0+ ?J ∗ z/? = maxk −zk = maxp∈ K −p z ) determines the relative interior of S, since it says that the recession function of J ∗ (see Rockafellar 1970, Theorem 8.5) must be the support function of K (see Rockafellar 1970, §4).
Lemma 6. Suppose that J ∗ satisfies the limit condition. Then the closure of S is K  Proof. It is a direct consequence of Theorem 13.3 and Corollary 10.5.1 in Rockafellar (1970).
Now, we suppose that J ∗ is a viscosity subsolution of H and satisfies the limit condition. By Corollary 12.2.2 in Rockafellar (1970) and the previous lemma, we deduce that J ∗ is necessarily of the form J ∗ z = supp∈ri K

Jp − p z , where J is a concave continuous function on ri K

 the relative interior of the convex-compact set K (by Theorem 10.1 in Rockafellar 1970). Now, we want to extend (continuously) J to all K . For that, we will use the fact that J ∗ is supposed to be a viscosity subsolution of H to prove that J is bounded below on ri K

and then we use the fact that K is a simplex in order to use the Extension Theorem 10.3 in Rockafellar (1970). (Note that Rockafellar 1970 deals with convex functions.) In fact, we need only to extend J by a finite upper-semi-continuous concave function on

K . Since here we deal with a simplex, this extension will be continuous. Lemma 7. ∀ p ∈ ri K

 Jp ≥ up . Proof. Let p ∈ ri K

and let z ∈ C + Jp . Then (by Theorem 23.5 in Rockafellar 1970), we have −p ∈ C − J ∗ z and J ∗ z = Jp − p z . Now, since J ∗ is a viscosity subsolution of E , and since −p ∈ C − J ∗ z , we deduce that J ∗ z ≥ up − p z  Hence, Jp ≥ up .




434

R. LARAKI

Corollary 3. J can be extended continuously to a concave continuous function on the whole K . Proof. The previous lemma implies that the concave continuous function on ri K

, J, is bounded below (by minp∈ K up ). Then one uses the fact that K is a simplex to apply Theorem 10.3 in Rockafellar (1970).
Now we will show that the fact that J ∗ is a viscosity solution of H implies that Kz t = tJ ∗ z/t = maxp∈ K tJp − p z  is the viscosity solution of E in U . Since equation E has a unique solution in U , we obtain uniqueness for H . For that, we need ± to compute Cloc Kz0  t0 . Lemma 8. ∀ z0 , ∀ t0 > 0 we have: − (i) Cloc Kz0  t0 = #q r  q ∈ C − J ∗ z0 /t0  r = J−q $. + (ii) Cloc Kz0  t0 = #q r  q =  J ∗ z0 /t0  r = J−q $. − Kz0  t0 ⊂ #q r  q ∈ C − J ∗ z0 /t0  r = Jq0 $. Proof. (i). First, let us prove that: Cloc ∗ We use the fact J is convex to deduce that its local subdifferential is the subgradient in convex analysis (see Barles 1995, §2.2). Actually,

q r ∈ C − Kz0  t0

Kz t − Kz0  t0 − z − z0  q − t − t0 r ≥0 z t →z0  t0

z − z0  + t − t0 

⇔ lim inf

tJ ∗ z/t − t0 J ∗ z0 /t0 − z − z0  q − t − t0 r ≥0 z t →z0  t0

z − z0  + t − t0 

⇔ lim inf

t0 J ∗ z/t0 − t0 J ∗ z0 /t0 − z − z0  q ≥0 z→z0 z − z0    z ⇔ q ∈ C−J∗ 0  t0 ⇒ lim inf

On the other hand, for all  (with  small), for all z0 ∈
K and t0 > 0, we have   
 z0 z0 K z0 +   t0 +  = max t0 +  Jp − z0 +   p p∈ K

t0 t0  

z0 p = max t0 Jp − z0  p +  Jp − p t0
   = max t0 Jp − z0  p + t0 Jp − z0  p p t0
      = max 1 + t0 Jp − z0  p p t0      max t0 Jp − z0  p = 1+ p t0      = max t0 Jp − z0  p + max t0 Jp − z0  p p t0 p   z = Kz0  t0 + J ∗ 0  t0 Thus, we conclude that

  Kz0 + z0 /t0  t0 +  − Kz0  t0

z = J∗ 0   t0

REPEATED GAMES WITH LACK OF INFORMATION

435

Thus,     Kz0 + z0 /t0  t0 +  − Kz0  t0 − z0 /t0  q − r z z = J ∗ 0 − 0  q − r  t0 t0 However, if q r ∈ C − Kz0  t0 , then, lim inf

→0

Kz0 + z0 /t0  t0 −  − Kz0  t0 − z0 /t0  q − r ≥ 0 

Thus, • by taking  > 0, we deduce that: J ∗ z0 /t0 − z0 /t0  q − r ≥ 0, • by taking  < 0 we deduce that: J ∗ z0 /t0 − z0 /t0  q − r ≤ 0. Hence, r = J ∗ z0 /t0 − z0 /t0  q . But since q ∈ C − J ∗ z0 /t0 by Fenchel-Legendre equality (Rockafellar 1970, Theorem 23.5), we deduce that  J∗

z0 t0



 = J−q −

 z0  −q  t0

which implies that r = J−q . − The proof of the inclusion #q r  q ∈ C − J ∗ z0 /t0  r = J−q $ ⊂ Cloc Kz0  t0 is straightforward as in the first part of the proof of Theorem 3 (above). The proof of (ii) is similar (we use the fact that when the local surdifferential (surgradient) of a convex function is nonempty at z, then this function is necessarily differentiable at z (see Rockafellar 1970, Theorem 25.1 and Barles 1995, §2.2).
Since J ∗ is a viscosity solution, we have: (1) q ∈ C − J ∗ z0 ⇒ J ∗ z0 − q z0 − u−q ≥ 0, (2) q =  J ∗ z0 ⇒ J ∗ z0 − q z0 − u−q ≤ 0. Now, J is a real-valued, upper-semicontinuous (in fact, continuous) concave function on K and since J ∗ is its Fenchel-Legendre conjugate, we have (see Rockafellar 1970, §23.5): 1 q ∈ C − J ∗ z0 ⇒ J ∗ z0 − q z0 − J−q = 0, 2 q =  J ∗ z0 ⇒ J ∗ z0 − q z0 − J−q = 0. Hence, (i) the previous Lemma 8, (1) and 1 imply that K is a viscosity sursolution of E , (ii) the previous Lemma 8, (2) and 2 imply that K is a viscosity subsolution of E . The limit condition for J ∗ implies the boundary condition for K. Now, since Kz t = maxp∈ K tJp − p z  with J is a continuous concave function on K , and K is compact, it is clear that K ∈ U . This ends the proof of Theorem 3.
Remark 5. • Let h = −f . Then h satisfies DMR in the viscosity sense. • Without the limit condition, H has infinitely many (linear) solutions: Mz0 = −z0  ? + u−? , ? ∈
K . • Theorem 3 and its proof are independent of the fact that u is the value of a game. Hence, u can be replaced by any continuous function and K by any convex-compact set. • Below (§5), we identify the viscosity solution of H as the Fenchel-Legendre conjugate of the restriction of u on K extended by − outside.

436

R. LARAKI

5. Translation of viscosity properties in the primal game, identification. Proposition 8. The viscosity solution of E is Bz t = maxp∈ K tv p − p z . Proof.

By Theorem 2, Proposition 7, Corollary 1, and Corollary 2, we deduce that   z Bz t = tf t  
z = t max v p − p p∈ K

t = max tv p − p z  p∈ K




Thus, the viscosity solution of E is of the form Kz t = maxp∈ K tJp − p z , where J is a concave continuous function on K . In order to determine v , we establish conditions on J for K to be the viscosity solution of E . We denote by J ∗ the Fenchel-Legendre conjugate of J J ∗ z = Kz 1 . We prove that K is a viscosity subsolution (resp. sursolution) if and only if J satisfies property P1 (resp. P2) below. Definition 4. p ∈ K is an extreme point of a function D on K if: for all ? ∈0 1 and for all pi i=1 2 ∈ K  p = ?p1 + 1 − ? p2 , and Dp = ?Dp1 + 1 − ? Dp2 imply p = p1 = p2 . D is the set of extreme points of D. Definition 5. Let D be a real-valued function on K . We define the properties P1 and P2 by: • P1: ∀ p0 ∈ D  Dp0 ≤ up0 . • P2: D· ≥ u· . Then we have the following: Theorem 4. (i) Kz t = maxp∈ K tvp − p z  is a viscosity subsolution of E if and only if J satisfies P1. (ii) Kz t = maxp∈ K tvp − p z  is a viscosity sursolution of E if and only if J satisfies P2. In particular, v is the unique concave continuous function on K satisfying P1 and P2. Corollary 4. v = Cav K u  where Cav K u is the smallest concave function on

K greater than u. Proof. Cav K u satisfies P1 and P2 and is continuous since K is a simplex (Theorem 10.3 in Rockafellar 1970).
The uniqueness of a concave continuous function satisfying P1 and P2 is easy to establish and does not need the use of the uniqueness of the viscosity solution (see Laraki 2001). To prove Theorem 4, we need the result of Lemma 8 where we proved that ∀ z0 , ∀ t0 > 0, we have: − Kz0  t0 = #q r  q ∈ C − J ∗ z0 /t0  r = J−q $. (i) Cloc + (ii) Cloc Kz0  t0 = #q r  q =  J ∗ z0 /t0  r = J−q $. We also need to prove some lemmas. The proof of Lemma 9 relies essentially on the argument used in the proof of Theorem 1.5.5, Chapter X in Hiriart-Urruty and Lemarechal (1993). Lemma 9. If  J ∗ z0 exists, then − J ∗ z0 ∈ J  Proof. Recall that J ∗ z0 = maxp Jp − p z0 . It is well known (Rockafellar 1970, Theorem 23.5) that p0 ∈ arg maxp Jp − p z0  if and only if: −p0 ∈ C − J ∗ z0 . Thus (Rockafellar 1970, Theorem 23.5 and Theorem 25.1),

437

REPEATED GAMES WITH LACK OF INFORMATION

J ∗ is differentiable at z0 if and only if arg maxp Jp − p z0  is reduced to only one element. Now, suppose that  J ∗ z0 = −p0 exists. Let ? ∈0 1 and suppose that ∃ pi i=1 2 ∈ K such that p0 = ?p1 + 1 − ? p2 and Jp0 = ?Jp1 + 1 − ? Jp2  Then, we deduce that Jp0 − p0  z0 = maxJp − p z0  p

= ?Jp1 − p1  z0  + 1 − ? Jp2 − p2  z0  Thus, necessarily we have pi i=1 2 ∈ arg maxp Jp − p z0 , which implies that p1 = p2 = p0 . Thus, p0 is an extreme point of J.
Lemma 10. Suppose that p0 ∈ J . Then there exists pn n∈N such that limn→ pn = p0 and there exists zn n∈N such that for any n  zn ∈ C + vpn and J ∗ is differentiable at zn . Proof. It is well known (see Rockafellar 1970, Theorem 25.6) that for a convex, continuous, real-valued function : on
K , one has C − :z0 = colimzn →z0  :zn , for all z0 ∈
K . Now, since p0 is an extreme point of J, it is necessarily an extreme point of C − J ∗ z0 = − arg maxp Jp − p z0 , with z0 ∈ C + Jp0 . Thus, we deduce (take : = J ∗ ) that there exist zn , such that  J ∗ zn → −p0 . Take pn = − J ∗ zn .
+ Kz0  t0  then Proof of Theorem 4. (i) Suppose that J satisfies P1. Let q r ∈ Cloc ∗ (by Lemma 8), q ∈  J z0 /t0 and r = J−q . Now by Lemma 9, we deduce that −q is an extreme point of J. Thus, r − u−q ≤ r − J−q ≤ 0 K is a viscosity subsolution of E . Now, suppose that K is a viscosity subsolution of E . Let p0 be an extreme point of J. By Lemma 10, we deduce that there exists pn n∈N such that limn→ pn = p0 and there exists zn n∈N such that for any n   J ∗ zn = −pn . Since K is a viscosity subsolution, we get that r = Jpn ≤ upn , and since J and u are continuous, we conclude by letting n go to infinity that Jp ≤ up : J satisfies P1. − Kz0  t0 ; then (by Lemma 8) we have (ii) Suppose that J satisfies P2. Let q r ∈ Cloc − ∗ q ∈ C J z0 /t0 and r = J−q  But, since J ≥ u, we conclude that r − u−q = J−q − u−q ≥ 0 K is a viscosity sursolution of E . Now, suppose that K is a viscosity sursolution. Let p ∈ K and let z ∈ C + Jp  Then, by Fenchel-Legendre duality (see Rockafellar 1970, Theorem 23.5), we deduce that −p ∈ − Kz 1 , and since K is a viscosity C − J ∗ z . By Lemma 8, we obtain that −p Jp

∈ Cloc sursolution, we conclude that Jp − up ≥ 0. Hence, J satisfies P2.
 6. Extension to generalized payoffs. Let : = :m m≥1 be a positive sequence with denote by G: p the game where the stream of payoffs is evaluated m≥1 :m = 1 and  through : Ep * +  m≥1 :m gm . The same argument as for G p implies that this game has a value v: p . The values of this class are called the compact values (Sorin 2002) because for the product topology, the strategy sets are compact and the payoff function is bilinear and jointly continuous. As above (§1), we define its dual G∗: z and we denote its value by w: z . Denote by 8: = tm m≥1 the subdivision of 0 1 induced by :. Namely, tm = m l=1 :l . Then, w: z is also the upper discrete value of the dual differential game associated to the subdivision 8: : Proposition 9. W8+: z 0 = w: z . Proof.

By definition of W8+: z 0 , we have:  W8+: z 0



= lim inf sup · · · inf sup g z − N →

y1

x1

yN

xN

N  l=1

 :l xl Ayl



438

R. LARAKI

On the other hand,

v: = lim v:N  N →

 where v:N p is the value of the primal game with payoff Ep * +  Nm=1 :m gm . However, by induction and using the same proof as De Meyer (1996) to establish the recursive structure Rn∗ , we prove that w:N , the Fenchel-Legendre conjugate of v:N , is precisely   N  N w: z = inf sup · · · inf sup g z − :l xl Ayl  y1

x1

yN

xN

l=1

limN → v:N ,

and since v: = we deduce that w: = limN → w:N , which ends the proof.
We  deduce that w: z converges uniformly, as : = supm :m  → 0 to f z with speed : and by duality, we obtain that v: converges uniformly with the same speed to Cav K u . Note that we give here an alternative proof of the fact that W+ z 0 = w z . The previous proof (Proposition 6, Lemma 2, and Lemma 4) uses a fixed point argument. Appendix. Let H 
K →
and b
K →
. Denote by E the Hamilton-Jacobi equation with boundary condition of the form  CD z t + H z Dz t

= 0 in
K ×0 T  E

Ct Dz 0 = bz  with D
K × 0 T  →
. + Given a function N O ⊂
M →
, the local surdifferential Cloc N and the local subdiffer− ential Cloc N at z0 are defined by (see Barles 1995, Definition 2.2, p. 17)     Nz − Nz0 − z − z0  q + M  Cloc Nz0 = q ∈
 lim sup ≤0  z − z0  z→z0 z∈O     Nz − Nz0 − z − z0  q − Cloc Nz0 = q ∈
M  lim inf ≥0  z→z0 z∈O z − z0  (Barles 1995, Theorem 2.2, p. 17) A viscosity solution of E is a continuous function D
K × 0 T  →
such that  + Dz t  r + H p ≤ 0 ∀ z ∈
K  ∀ t ∈0 T  ∀ p r ∈ Cloc  i

K − ii ∀ z ∈
 ∀ t ∈0 T  ∀ p r ∈ Cloc Dz t  r + H p ≥ 0   iii Dz 0 = bz  A continuous function D, satisfying (i) and (iii) is called a viscosity subsolution of E . A continuous function D satisfying (ii) and (iii) is called a viscosity sursolution of E . Notation • C
K is the space of continuous functions on
K . • BUC
K is the Banach space of bounded uniformly continuous functions on
K . 0 1 • C 0 1 
K (resp. Cb 
K

is the space of (resp. bounded) real-valued Lipschitz continuous functions. DD denotes the Lipschitz constant of D. n 
K (resp. Cbn 
K ) is the space of n times continuously differentiable functions • Cb

defined on
K (resp. which together with their n derivatives are  bounded). • For N
K →
such that C 2 N/Cxm Cxl exists D2 N = m l C 2 N/Cxm Cxl  . Theorem B (Crandall 1995, Theorem 3-1, p. 9). If H is continuous and b is uniformly continuous, then there exists a unique viscosity solution, Qz t  of E with the following properties: Q is uniformly continuous in z uniformly in t.

REPEATED GAMES WITH LACK OF INFORMATION

439

Theorem C (Crandall et al. 1984). If H ∈ C
K and b ∈ BUC
K , there exists a unique viscosity solution Q to E such that Q ∈ BUC
K × 0 T  . Proposition D (Crandall and Lions 1983). If H ∈ C
K and b is bounded and Lipschitz with constant L, then for all t ∈ 0 T  the bounded uniformly continuous viscosity solution of E, Qz t is Lipschitz in z with constant L. The F-conditions: Let F E , for 0 ≤ E ≤ T , be a mapping from BUC
K to itself: S → F E S satisfying: • F1: F 0 u = u. • F2: The mapping E → F E S is continuous. • F3: There is a constant C1 ≥ 0, such that F E S ≤ C1 E + S  • • • •

F4: F5: F6: F7:

F E S + c = F E S + c for every c ∈
. F E S1 − F E S2  ≤ S1 − S2   ∀ S1  S2 ∈ BUC
K . If S ∈ Cb0 1 
K then, F E S ∈ Cb0 1 
K and, DF E S ≤ DS. There exists a constant C2 such that for every N ∈ Cb2 
K

    F E N − N

  ≤ C2 E 1 + DN + D2 N   + H  N

  E 

Let 8F = t0  ' ' '  tN 8F be a finite subdivision of the interval 0 1 Define by induction the function Q8F by 

Q8F z 0 = u0 z  Q8F z t = F t − ti−1 u8F · ti−1

z if t ∈ti−1  ti 

Theorem E (Souganidis 1985b, p. 34). Let H ∈ C 0 1 and assume that for every E ≥ 0, F E satisfies F1 F2 ' ' '  F7. For b ∈ Cb0 1 
K , let Q be the viscosity of E and Q8F be as above. Then there exists C > 0 which depends only on b and Db such that: Q − Q8F  ≤ C8F 1/2 Theorem F (Bardi and Evans 1984). Lipschitz and convex, then

If H is continuous and if b is uniformly

Qz t = sup inf bq + p z − q − tH p  K p∈
K q∈


is the unique uniformly continuous viscosity solution of E . Acknowledgments. The author’s gratitude goes to Sylvain Sorin for motivating and supervising this work. His useful comments and advice have been very helpful. The author thanks J-B. Hiriart-Urruty, C. Imbert, and X. Spinat for useful references and would like to express gratitude to the referees for their interesting remarks. Part of this work was done when the author was also affiliated at Modal’X, UFR-SEGMI, Université Paris 10 Nanterre.

440

R. LARAKI

References Aumann, R. J., M. Maschler, R. B. Stearns. 1995. Repeated Games with Incomplete Information. M.I.T. Press, Cambridge, MA. Bardi, M., L. C. Evans. 1984. On Hopf’s formulas for solutions of Hamilton-Jacobi equations. Nonlinear Anal., Theory, Methods Appl. 8 1373–1381. Barles, G. 1995. Solution de Viscosité des Équations de Hamilton-Jacobi. Springer-Verlag, Berlin. Barron, E. N., L. C. Evans, R. Jensen. 1984. Viscosity solutions of Isaacs’s equations and differential games with Lipschitz controls. J. Differential Equations 53 213–233. Benoist, J., J-B. Hiriart-Urruty. 1996. What is the subdifferential of the closed convex hull of a function? SIAM J. Math. Anal. 27 1661–1679. Blackwell, D. 1956. An analog of the minmax theorem for vector payoffs. Pacific J. Math. 65 1–8. Crandall, D., et al. 1995. Viscosity solutions and applications. Lecture Notes in Mathematics, vol. 1660. Springer, Berlin, Germany. , L. C. Evans, P. L. Lions. 1984. Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. 282 487–502. , P. L. Lions. 1983. Viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. 277 1–42. De Meyer, B. 1996. Repeated games and partial differential equations. Math. Oper. Res. 21 209–236. , D. Rosenberg. 1999. “Cav u” and the dual game. Math. Oper. Res. 24 619–626. Fleming, W. 1961. The convergence problem for differential games. J. Math. Anal. Appl. 3 102–116. . 1964. The convergence problem for differential games II. Advances in Game Theory, vol. 52, Annals of Mathematics Studies. Princeton University Press, Princeton, NJ, 102–116. Hiriart-Urruty, J-B., C. Lemarechal. 1993. Convex Analysis and Minimization Algorithms. Springer-Verlag, Berlin. Laraki, R. 2001. Variational inequalities, system of functional equations and incomplete information repeated games. SIAM J. Control Optim. 40(2) 516–524. Mertens, J. F., S. Sorin, and S. Zamir. 1994. Repeated games. Core Discussion Paper, 9420, 9421, and 9422. Université catholique de Louvain, Louvain-la-Neuve, Belgium. Rockafellar, R. T. 1970. Convex Analysis. Princeton University Press, Princeton, NJ. Rosenberg, D. 1998. Duality and Markovian strategies. Internat. J. Game Theory 27 577–597. Sion, M. 1958. On general minmax theorems. Pacific J. Math. 8 171–176. Sorin, S. 1984. “Big Mach” with lack of information on one side. Part I. Internat. J. Game Theory 13 201–255. . 2002. A First Course on Zero-Sum Repeated Games. Springer, Berlin. Souganidis, P. E. 1985a. Max-min representation and product formulas for the viscosity solutions of HamiltonJacobi equations with applications to differential games. Nonlinear Anal. Theory, Methods Appl. 9 217–257. . 1985b. Approximation schemes for viscosity solutions of Hamilton-Jacobi equations. J. Differential Equations 59 1–43. Vieille, N. 1992. Weak approachability. Math. Oper. Res. 17 781–791. R. Laraki: CNRS Laboratoire d’Econométrie de l’École Polytechnique, 1 rue Descartes, 75005 Paris, France; e-mail: [email protected]