Periodic homogenization of Hamilton-Jacobi-Bellman equations Naoyuki ICHIHARA∗

Abstract This paper is concerned with periodic homogenization of Hamilton-JacobiBellman equations arising in stochastic control. Our proof is based on probabilistic arguments using backward stochastic differential equations. As a byproduct, we are able to calculate the convergence rate of solutions to that of the homogenized (effective) equation. We also discuss a stochastic control interpretation for the effective equation.

1

Introduction.

This paper is concerned with the periodic homogenization of the following fully nonlinear second-order parabolic equation −ut + H

¡x ¢ , x, u, Du, D2 u = 0 , ε

in (0, T ) × Rd

(1)

with a given terminal function u(T, · ) = h ∈ BUC(Rn ), where ε > 0 denotes a scale parameter. The Hamiltonian H is assumed to be Zd -periodic with respect to the first variable. In this paper, we are particularly interested in the case where H is of the form ½ ¾ 1 H(η, x, y, p, X) = sup − aij (η, α)Xij − bi (η, α)pi − f (η, x, y, p, α) . (2) 2 α∈U ∗

E-mail: [email protected]. Graduate School of Natural Science and Technology (Faculty of Environmental Science and Technology), Okayama University. Keywords: Homogenization, backward stochastic differential equations, Hamilton-JacobiBellman equations, rate of convergence.

1

Here the summation convention for the repeated indices i and j is used in the righthand side. More precise conditions on the coefficients a, b and f and the control region U will be stated in the next section. The main purpose of this paper is to prove the following convergence result (periodic homogenization): Theorem 1.1. Let {uε ; ε > 0} be the family of viscosity solutions to (1)-(2). Then, as ε goes to zero, it converges, locally uniformly in [0, T ] × Rd , to a unique viscosity solution u of −ut + H(x, u, Du, D2 u) = 0 ,

in

(0, T ) × Rd

(3)

with u(T, · ) = h. Here, the effective Hamiltonian H = H(x, y, p, X) is given by ½ ¾ 1 H(x, y, p, X) = sup − aij (µ)Xij − bi (µ)pi − f (x, y, p, µ) (4) 2 µ∈U for suitable control space U and functions a, b and f that will be specified in Section 4. Periodic homogenization of this type has been largely studied in the theory of viscosity solution (see [6] for the notion of viscosity solution). One of the most powerful tools to prove it is the so-called perturbed test function method initiated by Evans [8]. We refer to [1], [2] and [9] for further development of this approach. On the other hand, it is worth studying this problem in the probabilistic framework, for equation (1)-(2) is closely related to several important and interesting stochastic control problems. There are many literatures on the periodic homogenization of second-order partial differential equations. See for instance [3], [4], [7] and [11] that use the probabilistic approach based on backward stochastic differential equations (BSDEs), a variant of stochastic differential equations (SDEs). We will discuss this point in Section 3. In this paper, we aim at developing the above BSDE approach for fully nonlinear equations such as Hamilton-Jacobi-Bellman equations. One of the main advantages of this approach compared to the viscosity solution (analytic) method is that we are able to get intuitive understanding of homogenization phenomena by looking at stochastic processes behind them. This also allows one to obtain precise estimates (e.g. the rate of convergence) of solutions, as well as the stochastic control interpretation for the homogenized equation (3). However, the price to pay is that we 2

require strong regularity property for solutions which is not needed in the viscosity solution method. We finally remark that various extensions of our results to more general fully nonlinear equations are possible with the aid of BSDE arguments. We refer to [5] and [10] for more information on this subject.

2

Assumptions.

Throughout this paper, we assume the following conditions. (A1) U is a compact metric space. (A2) a ∈ BUC(Rd × U ; Rd ⊗ Rd ) and b ∈ BUC(Rd × U ; Rd ) are Zd -periodic in their first variable. Moreover, they are Lipschitz continuous with respect to the first variable uniformly in α ∈ U . (A3) The (d × d)-matrix valued function a is uniformly elliptic. (A4) f = f (η, x, y, p, α) ∈ BUC(Rd × Rd × R × Rd × U ) is Zd -periodic in η and δ-H¨older continuous with resect to (η, x) uniformly in the other variables. Moreover, there exists an L > 0 such that f is Lipschitz continuous in p with Lipschitz constant L and the mapping y 7→ f (η, x, y, p, α) − Ly is non-increasing. (A5) h ∈ C 2+δ (Rd ), where C 2+δ (Rd ) denotes the H¨older space of exponent 2 + δ. Remark 2.1. (a) Under (A1)-(A5), for every ε > 0, equation (1) has a unique δ classical solution in the H¨older space C 1+ 2 ,2+δ ([0, T ] × Rd ). (b) Suppose that U in (4) is compact and that the coefficients a, b and f in (4) satisfy (A1)-(A5) in place of a, b and f , respectively. Then, equation (3) has a unique classical solution in the same H¨older space as (a).

3

Probabilistic approach.

We now construct, for each ε > 0, a stochastic control problem whose value function coincides with the solution of (1). Let (Ω, F, P ) be a complete probability space equipped with a filtration F := {Fs }0≤s≤T and a d-dimensional F-Brownian motion {Ws }0≤s≤T . Let us denote by A the set of U -valued F-progressively measurable processes {αs }0≤s≤T . For given (t, x) ∈ [0, T ] × Rd and α ∈ A, we consider the forward SDE dXsε,α = b(

Xsε,α X ε,α , αs ) ds + σ( s , αs ) dWs , ε ε 3

t ≤ s ≤ T,

(5)

with initial condition Xtε,α = x and the backward SDE dYsε,α = f (

Xsε,α ε,α ε,α ε,α X ε,α , Xs , Ys , Zs , αs ) ds − Zsε,α σ( s , αs ) dWs , t ≤ s ≤ T, ε ε

(6)

with terminal condition YTε,α = h(XTε,α ). Here, σ ∈ BUC(Rd × U ; Rd ⊗ Rd ) is such that a = σσ ∗ . Remark that, under (A2) and (A3), we may assume that σ is Lipschitz continuous with respect to η uniformly in α ∈ U . Note also that system (5)-(6) has a unique F-progressively measurable solution (X ε , Y ε , Z ε ). Now, our value function is defined by uε (t, x) := ess-inf Ytε,α a∈A

(7)

where ”ess-inf” in (7) is taken over all U -valued F-progressively measurable processes. Remark. The value Ytε,α is actually a non-random quantity although it is, a priori, Ft -measurable. Thus, ”ess-inf” in (7) can be replaced by ”inf”. Proposition 3.1. For each ε > 0, the function uε (t, x) defined by (7) is a unique viscosity solution of (1). Remark. The value function (7) is a generalization of usual ones in stochastic control. To see this, let us assume that f depends only on (η, α). Then, we have Z T Z T Xsε,α X ε,α ε,α ε,α Yt = h(XT ) + f( , αs ) ds − Zsε,α σ( s , αs ) dWs . ε ε t t By taking expectation in both sides, we get ·Z T ¸ Xsε,α ε,α ε,α Yt = E f( , αs ) ds + h(XT ) , ε t which is nothing but the classical cost functional. Here, we have used the fact that Ytε,α is deterministic. ε,α

We now set Y s := Ysε,α − u(s, Xsε,α ), where u is the solution of (3). Then, we can show the following theorem which easily deduces Theorem 1.1. Theorem 3.2. Assume (A1)-(A5). Then, for every compact subset Q of [0, T ]×Rd , there exists a constant C > 0 independent of ε > 0 such that ¯ 2δ ε,α ¯ (8) sup ¯ inf Y t ¯ ≤ C ε 2+δ , (t,x)∈Q α∈A

where δ ∈ (0, 1) is the H¨older exponent appearing in (A4). 4

We refer to [10] for the proof of this theorem. The idea is as follows. For each fixed (x, y, p, X), we consider the cell problem H = H(η, x, y, p, X + D2 v(η))

in Td ,

(9)

where Td stands for the d-dimensional unit torus. Note that unknowns in (9) are the constant H = H(x, y, p, X) and the function v(η) = v(η; x, y, p, X) and that H defines the effective Hamiltonian in equation (3). We next set v(η, s, x) := v(η; u(s, x), Du(s, x), D2 u(s, x)) ε,α

and apply Ito’s formula to Ysε,α − u(s, Xsε,α ) − ε2 v( Xsε , s, Xsε,α ) in order to show lim inf E[Ysε,α − u(s, Xsε,α ) − ε2 v(

ε→0 α∈A

Xsε,α , s, Xsε,α )] = 0, ε

from which we get the convergence uε (t, x) −→ u(t, x) as ε → 0. Unfortunately, this procedure is too naive to justify even if u is sufficiently smooth. The reason is the function v(η) = v(η; x, y, p, X) is not differentiable with respect to (x, y, p, X). Nevertheless, by freezing the slow variable X ε,α and then by looking at detailed asymptotics of the fast variable ε−1 X ε,α as ε → 0, we can develop the ”local” argument to establish estimate (8). The drawback of this probabilistic procedure is that u should be of C 1,2 -class so that Ito’s formula is justified. That is why we need strong regularity assumption on the solution of effective equation (3).

4

Stochastic interpretation for the effective equation.

We next consider a limiting stochastic control problem corresponding to the homogenized equation (3). For this purpose, we construct a new control space. Let M = M(Td × U ) be the set of probability measures on Td × U equipped with the topology of weak convergence of probability measures. Note that M is a compact metrizable space and we have the natural injection Td × U ,→ M. For each k ∈ C(Td × U ), we define its extension k ∈ C(M) by Z k(µ) := k(η, α) µ(dηdα) , µ ∈ M. Td ×U

ˆ := {Fˆs }s≥0 , {W ˆ F, ˆ Pˆ ; F ˆ s }s≥0 ) be any probability space with filtration and Let (Ω, ˆ Brownian motion. We denote by Aˆ the set of U -valued F-adapted processes {αs }s≥0 5

ˆ F, ˆ Pˆ ). For a given control process α ∈ A, ˆ we consider the SDE on (Ω, ˆs , dηsα = b(ηsα , αs ) ds + σ(ηsα , αs ) dW

η0α = η ∈ Rd .

Then, we can associate a family of probability measures { µαt ; t ≥ 0 } ⊂ M such that Z T hZ T i α ˆ k(µt ) dt = E k(ηtα , αt ) dt for all k ∈ C(Td × U ) and T > 0. 0

0

ˆ we set U := S ˆ U(α) , where For α ∈ A, α∈A U(α) :=

©

¯ 1 µ ∈ M ¯ µ = lim n→∞ Tn

Z

Tn 0

ª µαt dt for some Tn → ∞ (n → ∞) .

Note that U is a non-empty and compact subset of M. Now, let a, b ∈ BUC(M) and f ∈ BUC(Rd × R × Rd × M) be the extended functions of a, b and f , respectively. Let (Ω, F, P ; F, {Ws }0≤s≤T ) be a filtered probability space with Brownian motion. For a given U-valued F-progressively measurable process µ, we consider the following system of forward and backward SDEs: dXsµ = b(µs ) ds + θ(µs ) dWs ,

Xtµ = x ∈ Rd ,

dYsµ = f (Xsµ , Ysµ , Zsµ , µs ) ds − Zsµ θ(µs ) dWs ,

YTµ = h(XTµ ) ,

(10) (11)

where θ ∈ BUC(M) is taken so that a = θθ∗ , which is in general different from the extended function σ of the original σ. We define the value function by u(t, x) := ess-inf Ytµ , µ

(12)

where ”ess-inf” is taken over all U-valued F-progressively measurable processes. Proposition 4.1. The function u(t, x) defined by (12) is a unique viscosity solution of the homogenized equation (3)-(4). A complete proof of this proposition can be found in [5].

References [1] Alvarez O, Bardi M (2001) Viscosity solutions methods for singular perturbations in deterministic and stochastic control. SIAM J Control Optim 40(4):11591188 6

[2] Alvarez O, Bardi M (2003) Singular perturbations of nonlinear degenerate parabolic PDEs: a general convergence result. Arch Ration Mech Anal 170(1):17–61 [3] Buckdahn R, Hu Y (1998) Probabilistic approach to homogenization of quasilinear parabolic PDEs with periodic structure. Nonlinear Anal TMA & Applications 32:609-619 [4] Buckdahn R, Hu Y, Peng S, (1999) Probabilistic approach to homogenization of viscosity solutions of parabolic PDEs. Nonlinear Differential Equations Appl 6:395-411 [5] Buckdahn R, Ichihara N (2005) Limit theorem for controlled backward SDEs and homogenization of Hamilton-Jacobi-Bellman equations. Applied Mathematics and Optimization 51(1):1-33 [6] Crandall MG, Ishii H, Lions P-L (1992) Use’s guide to viscosity solutions of second order partial differential equations. Bull Amer Math Soc (N.S.) 27:1-67 [7] Delarue F, (2004) Auxiliary SDEs for homogenization of quasilinear PDEs with periodic coefficients. Ann Probab 32(3B):2305-2361 [8] Evans LC (1989) The perturbed test function method for viscosity solutions of nonlinear PDEs. Proc Roy Soc Edinburgh Sect A 111:359-375 [9] Evans LC (1992) Periodic homogenization of certain fully nonlinear partial differential equations. Proc Roy Soc Edinburgh Sect A 120:245-265 [10] Ichihara N (2005) A stochastic representation for fully nonlinear PDEs and its application to homogenization, Journal of Mathematical Sciences, the University of Tokyo 12(3):467-492 [11] Pardoux E (1999) Homogenization of linear and semilinear second order parabolic PDEs with periodic coefficients. J Funct Anal 167:498-520

7