Lectures on Stochastic control and applications in finance
Huyˆen PHAM University Paris Diderot, LPMA Institut Universitaire de France and CREST-ENSAE
[email protected] http://www.proba.jussieu.fr/pageperso/pham/pham.html
Autumn school on “Stochastic control problems for FBSDEs and Applications” Marrakech, December 1-11, 2010
Stochastic Control and applications in finance
Abstract. The aim of these lectures is to present an introduction to stochastic control, a classsical topic in applied mathematics, which has known important developments over the last years inspired especially by problems in mathematical finance. We give an overview of the main methods and results in this area. We first present the standard approach by dynamic programming equation and verification, and point out the limits of this method. We then move on to the viscosity solutions approach: it requires more theory and technique, but provides the general mathematical tool for dealing with stochastic control in a Markovian context. The last lecture is devoted to an introduction to the theory of Backward stochastic differential equations (BSDEs), which has emerged as a major research topic with significant contributions in relation with stochastic control beyond the Markovian framework. The various methods presented in these lectures will be illustrated by several applications arising in economics and finance. Lecture 1 : Classical approach to stochastic control problem Lecture 2 : Viscosity solutions and stochastic control Lecture 3 : BSDEs and stochastic control
References for these lectures:
• H. Pham (2009): Continuous-time stochastic control and optimization with financial applications, Series SMAP, Springer. • I. Kharroubi, J. Ma, H. Pham and J. Zhang (2010): “Backward stochastic differential equations with constrained jumps and quasi-variational inequalities”, Annals of Probability, Vol. 38, 794-840.
Lecture 1 : Classical approach to stochastic control problem
• Introduction • Controlled diffusion processes • Dynamic Programming Principle (DPP) • Hamilton-Jacobi-Bellman (HJB) equation • Verification theorem • Applications : Merton portfolio selection (CRRA utility functions and general utility functions by duality approach), Merton portfolio/ consumption choice • Some other classes of stochastic control
I. Introduction • Basic structure of stochastic control problem • Dynamic system in an uncertain environment: - filtered probability space (Ω, F, F = (Ft ), P): uncertainty and information - state variables X = (Xt ): F-adapted stochastic process representing the evolution of the quantitative variables describing the system • Control: a process α = (αt ) whose value is decided at time t in function of the available information Ft , and which can influence the dynamics of the state process X. • Performance/criterion: optimize over controls a functional J(X, α), e.g. Z T J(X, α) = E f (Xt , αt )dt + g(XT ) on a finite horizon 0
or Z J(X, α) = E
∞ −βt
e
f (Xt , αt )dt
on an infinite horizon
0
I Various and numerous applications in economics and finance I In parallel, problems in mathematical finance → new developments in the theory of stochastic control
• Solving a stochastic control problem • Basic goal: find the optimal control (which achieves the optimum of the objective functional) if it exists and the value function (the optimal objective functional) • Tractable characterization of the value function and optimal control → - if possible, explicit solutions - otherwise: qualitative description and quantitative results via numerical solutions
• Mathematical tools • Dynamic programming principle and stochastic calculus → - PDE characterization in a Markovian context - BSDE in general I Stochastic control is a topic at the interface between probability, stochastic analysis and PDE.
II. Controlled diffusion processes • Dynamics of the state variables in Rn : dXs = b(Xs , αs )ds + σ(Xs , αs )dWs ,
(1)
W d-dimensional Brownian motion on (Ω, F, F = (Ft ), P). - The control α = (αt ) is an F-adapted process, valued in A subset of Rm , and satisfying some integrability conditions and/or state constraints → A set of admissible controls. - Given α ∈ A, (t, x) ∈ [0, T ] × Rn , we denote by X t,x = X t,x,α the solution to (1) starting from x at t. • Performance criterion (on finite horizon) Given a function f from Rn × A into R, and a function g from Rn into R, we define the objective functional: Z T t,x t,x J(t, x, α) = E f (Xs , αs )ds + g(XT ) , (t, x) ∈ [0, T ] × Rn , α ∈ A, t
and the value function: v(t, x) = sup J(t, x, α). α∈A
•α ˆ ∈ A is an optimal control if: v(t, x) = J(t, x, α ˆ ). • A process α in the form αs = a(s, Xst,x ) for some measurable function a from [0, T ] × Rn into A is called Markovian or feedback control.
III. Dynamic programming principle Bellman’s principle of optimality “An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision” (See Bellman, 1957, Ch. III.3) Mathematical formulation of the Bellman’s principle or Dynamic Programming Principle (DPP) The usual version of the DPP is written as Z θ t,x t,x v(t, x) = sup E f (Xs , αs )ds + v(θ, Xθ ) , α∈A
t
for any stopping time θ ∈ Tt,T (set of stopping times valued in [t, T ]).
(2)
• Stronger version of the DPP In a stronger and useful version of the DPP, θ may actually depend on α in (2). This means: Z
θ
v(t, x) = sup sup E α∈A θ∈Tt,T
f (Xst,x , αs )ds + f (Xst,x , αs )ds
v(θ, Xθt,x )
t
Z = sup inf E α∈A θ∈Tt,T
t
v(θ, Xθt,x )
θ
+
⇐⇒ (i) For all α ∈ A and θ ∈ Tt,T : Z v(t, x) ≥ E t
θ
f (Xst,x , αs )ds + v(θ, Xθt,x ) .
(ii) For all ε > 0, there exists α ∈ A such that for all θ ∈ Tt,T Z θ t,x t,x v(t, x) − ε ≤ E f (Xs , αs )ds + v(θ, Xθ ) . t
.
Proof of the DPP. (First part) 1. Given α ∈ A, we have by pathwise uniqueness of the flow of the SDE for X, the Markovian structure θ,Xθt,x
Xst,x = Xs
,
s ≥ θ,
for any θ ∈ Tt,T . By the law of iterated conditional expectation, we then get Z θ t,x J(t, x, α) = E f (s, Xst,x , αs )ds + J(θ, Xθ , α) , t
Since J(., ., α) ≤ v, and θ is arbitrary in Tt,T , this implies Z θ J(t, x, α) ≤ inf E f (s, Xst,x , αs )ds + v(θ, Xθt,x ) θ∈Tt,T t Z θ ≤ sup inf E f (s, Xst,x , αs )ds + v(θ, Xθt,x ) . α∈A θ∈Tt,T
t
=⇒
Z v(t, x) ≤ sup inf E α∈A θ∈Tt,T
t
θ
f (s, Xst,x , αs )ds +
v(θ, Xθt,x )
.
(3)
Proof of the DPP. (Second part) 2. Fix some arbitrary control α ∈ A and θ ∈ Tt,T . By definition of the value functions, for any ε > 0 and ω ∈ Ω, there exists αε,ω ∈ A, which is an ε-optimal control for t,x v(θ(ω), Xθ(ω) (ω)), i.e. t,x t,x (ω), αε,ω ). (ω)) − ε ≤ J(θ(ω), Xθ(ω) v(θ(ω), Xθ(ω)
(4)
Let us now define the process ( α ˆ s (ω) =
αs (ω),
s ∈ [0, θ(ω)]
αsε,ω (ω), s ∈ [θ(ω), T ].
→ Delicate measurability questions! By measurable selection results, one can show that α ˆ is F-adapted, and so α ˆ ∈ A. By using again the law of iterated conditional expectation, and from (4): Z θ t,x t,x ε v(t, x) ≥ J(t, x, α ˆ) = E f (s, Xs , αs )ds + J(θ, Xθ , α ) t Z θ t,x t,x ≥ E f (s, Xs , αs )ds + v(θ, Xθ ) − ε. t
Since α, θ and ε are arbitrary, this implies Z θ v(t, x) ≥ sup sup E f (s, Xst,x , αs )ds + v(θ, Xθt,x ) . α∈A θ∈Tt,T
t
IV. Hamilton-Jacobi-Bellman (HJB) equation The HJB equation is the infinitesimal version of the dynamic programming principle: it describes the local behavior of the value function when we send the stopping time θ in the DPP (2) to t. The HJB equation is also called dynamic programming equation. Formal derivation of HJB We assume that the value function is smooth enough to apply Itˆo’s formula, and we postpone integrability questions. I For any α ∈ A, and a controlled process X t,x , apply Itˆo’s formula to v(s, Xst,x ) between s = t and s = t + h: Z t,x v(t + h, Xt+h ) = v(t, x) +
t
t+h
∂v αs + L v (s, Xst,x )ds ∂t
+ (local) martingale,
where for a ∈ A, La is the second-order operator associated to the diffusion X with constant control a: 1 La v = b(x, a).Dx v + tr σ(x, a)σ 0 (x, a)Dx2 v . 2
I Plug into the DPP: t+h
Z sup E α∈A
t
∂v + Lαs v (s, Xst,x ) + f (Xst,x , αs )ds ∂t
= 0.
I Divide by h, send h to zero, and “obtain” by the mean-value theorem, the so-called HJB equation: ∂v + sup [La v + f (x, a)] = 0, (t, x) ∈ [0, T ) × Rn . ∂t a∈A
(5)
Moreover, if v is continuous at T , we have the terminal condition v(T − , x) = v(T, x) = g(x),
x ∈ Rn .
Classical approach to stochastic control: • Show if possible the existence of a smooth solution to HJB, or even better obtain an explicit solution • Verification step: prove that this smooth solution to HJB is the value function of the stochastic control problem, and obtain as a byproduct the optimal control. Remark. In the classical verification approach, we don’t need to prove the DPP, but “only” to get the existence of a smooth solution to the HJB equation.
V. Verification approach Theorem Let w be a function in C 1,2 ([0, T ] × Rn ), solution to the HJB equation: ∂w (t, x) + sup La w(t, x) + f (x, a) = 0, (t, x) ∈ [0, T ) × Rn , ∂t a∈A w(T, x) = g(x), x ∈ Rn . (and satisfying eventually additional growth conditions related to f and g). Suppose there exists a measurable function a ˆ(t, x), (t, x) ∈ [0, T ) × Rn , valued in A, attaining the supremum in HJB i.e. sup La w(t, x) + f (x, a) = Laˆ(t,x) w(t, x) + f (x, a ˆ(t, x)), a∈A
such that the SDE dXs = b(Xs , a ˆ(s, Xs ))ds + σ(Xs , a ˆ(s, Xs ))dWs ˆ st,x , given an initial condition Xt = x, and the admits a unique solution, denoted by X ˆ st,x ) t ≤ s ≤ T } lies in A. Then, process α ˆ = {ˆ a(s, X w = v, and α ˆ is an optimal feedback control.
Proof of the verification theorem. (First part) 1. Suppose that w is a smooth supersolution to the HJB equation: −
∂w (t, x) − sup La w(t, x) + f (x, a) ≥ 0, (t, x) ∈ [0, T ) × Rn , ∂t a∈A w(T, x) ≥ g(x), x ∈ Rn .
(6) (7)
I For any α ∈ A, and a controlled process X t,x , apply Itˆo’s formula to w(s, Xst,x ) between s = t and s = T ∧ τn , and take expectation: Z T ∧τn ∂w t,x αs t,x E w(T ∧ τn , XT ∧τn ) = w(t, x) + E + L w (s, Xs )ds ∂t t where (τn ) is a localizing sequence of stopping times for the local martingale appearing in Itˆo’s formula. I Since w is a supersolution to HJB (6), this implies: Z T ∧τn t,x t,x E w(T ∧ τn , XT ∧τn ) + E f (Xs , αs )ds ≤ w(t, x). t
By sending n to infinity, and under suitable integrability conditions, we get: Z T E w(T, XTt,x ) + E f (Xst,x , αs )ds ≤ w(t, x). t
I Since w(T, .) ≥ g, and α is arbitrary, we obtain Z T t,x t,x v(t, x) = sup E f (Xs , αs )ds + g(XT ) ≤ w(t, x). α∈A
t
Proof of the verification theorem. (Second part) 2. Suppose that the supremum in HJB equation is attained: −
∂w (t, x) − Laˆ(t,x) w(t, x) + f (x, a ˆ(t, x)) = 0, (t, x) ∈ [0, T ) × Rn , ∂t w(T, x) = g(x), x ∈ Rn .
(8) (9)
ˆ st,x ) for the feedback control α ˆ . By same arguments I Apply Itˆo’s formula to w(s, X as in the first part, we have now the equality (after an eventual localization): Z T ∂w t,x α ˆs t,x ˆ s )ds + w(T, X ˆ ) w(t, x) = −E + L w (s, X T ∂t t Z T t,x t,x ˆs , α ˆ ) (≤ v(t, x)). = E f (X ˆ s )ds + g(X T t
I Together with the first part, this proves that w = v and α ˆ is an optimal feedback control.
Probabilistic formulation of the verification approach The analytic statement of the verification theorem has a probabilistic formulation: Suppose that the measurable function w on [0, T ] × Rn satisfies the two properties: • for any control α ∈ A with associated controlled process X, the process Z t w(t, Xt ) + f (Xs , αs )ds is a supermartingale
(10)
0
• there exists a control α ˆ ∈ A with associated controlled process X, such that the process ˆt) + w(t, X
Z
t
ˆs, α f (X ˆ s )ds
is a martingale.
(11)
0
Then, w = v, and α ˆ is an optimal control. Remark. Notice that in the probabilistic verification approach, we do not need smoothness of w, but we require a supermartingale property. In the analytic verification approach, the smoothness of w is used for applying Itˆo’s formula to w(t, Xt ). This allows us to derive the supermartingale property as in (10), which is in fact the key feature for proving that w ≥ v, and then w = v with the martingale property (11).
VI. Applications 1. Merton portfolio selection in finite horizon An agent invests at any time t a proportion αt of his wealth X in a stock of price S and 1 − αt in a bond of price S 0 with interest rate r. The investor faces the portfolio constraint that at any time t, αt is valued in A closed convex subset of R. I Assuming a Black-Scholes model for S (with constant rate of return µ and volatility σ > 0), the dynamics of the controlled wealth process is: Xt αt Xt (1 − αt ) 0 dSt + dSt St St0 = Xt (r + αt (µ − r)) dt + Xt αt σdWt .
dXt =
• The preferences of the agent is described by a utility function U : increasing and concave function. The performance of a portfolio strategy is measured by the expected utility from terminal wealth → Utility maximization problem at a finite horizon T :
v(t, x) = sup E[U (XTt,x )], (t, x) ∈ [0, T ] × (0, ∞). α∈A
→ Standard stochastic control problem
HJB equation for Merton’s problem
h 1 2 2 2 i vt + rxvx + sup a(µ − r)xvx + x a σ vxx = 0, (t, x) ∈ [0, T ) × (0, ∞) 2 a∈A v(T, x) = U (x), x > 0.
• The case of CRRA utility functions: xp U (x) = , p
p < 1, p 6= 0
→ Relative Risk Aversion: −xU 00 (x)/U 0 (x) = 1 − p. I We look for a candidate solution to HJB in the form w(t, x) = ϕ(t)U (x). Plugging into HJB, we see that ϕ should satisfy the ODE: ϕ0 (t) + ρϕ(t) = 0,
φ(T ) = 1,
where 1 ρ = rp + p sup a(µ − r) − a2 (1 − p)σ 2 , 2 a∈A → ϕ(t) = eρ(T −t) .
I The value function is equal to v(t, x) = eρ(T −t) U (x), and the optimal control is constant (in proportion of wealth invested) 1 a ˆ = arg max a(µ − r) − a2 (1 − p)σ 2 . a∈A 2 When A = R (no portfolio constraint), the values of ρ and a ˆ are explicitly given by ρ =
(µ − r)2 p + rp. 2σ 2 1 − p
and a ˆ =
µ−r , − p)
σ 2 (1
• General utility functions: U is C 1 , strictly increasing and concave on (0, ∞), and satisfies the Inada conditions: U 0 (0) = ∞,
U 0 (∞) = 0.
I Convex conjugate of U : U˜ (y) := sup[U (x) − xy] = U (I(y)) − yI(y),
y > 0,
x>0
where I := (U 0 )−1 = −U˜ 0 . • Assume that A = R (no portfolio constraint and complete market) and for simplicity r = 0 so that HJB is also written as 1 µ2 vx2 vt − = 0, 2 σ 2 vxx with a candidate for the optimal feedback control: a ˆ(t, x) = −
µ vx . σ 2 x2 vxx
Recall the terminal condition: v(T, x) = U (x).
→ Fully nonlinear second order PDE → But remarkably, it can be solved explicitly by convex duality!
• Introduce the convex conjugate of v, also called dual value function: v˜(t, y) = sup[v(t, x) − xy],
y > 0.
x>0
↔ change of variables: y = vx and x = −˜ vy . I v˜ satisfies the linear parabolic Cauchy problem: 1 µ2 2 y v˜yy = 0 v˜t + 2 σ2 v˜(T, y) = U˜ (y). From Feynman-Kac formula, v˜ is represented as v˜(t, y) = E U˜ (yYTt ) , where Y t is the solution to µ dYst = −Yst dWs , σ Remark.
YTt
Ytt = 1.
i h = E dQ/dP Ft is the density of the risk-neutral probability measure
Q, under which S is a martingale: dSt = St σdWtQ ,
• The primal value function is obtained by duality relation: v(t, x) = inf v˜(t, y) + xy , y>0
x > 0.
I From the representation of v˜, we get: n o t ˜ v(t, x) = inf E U (yYT ) + xy y>0
(12)
Recalling that U˜ 0 = −(U 0 )−1 =: I, the infimum in (12) is attained at yˆ = yˆ(t, x) s.t. E YTt I(ˆ y YTt ) = x,
(saturation budget constraint)
(13)
and we have h i t t t ˜ v(t, x) = E U (ˆ y YT ) + yˆYT I(ˆ y YT ) . Recalling that the supremum in U˜ is attained at x = I(y), i.e. U˜ (y) = U (I(y))−yI(y), we obtain: h i t,x ˆ v(t, x) = E U (XT ) ,
ˆ t,x = I(ˆ with X y YTt ). T
(14)
I Consider now the strictly positive Q-martingale process: i h t,x Q t ˆ s := E I(ˆ X y YT ) Fs , t ≤ s ≤ T. ˆ tt,x = x. - From the saturation budget constraint (13), we have X - From the martingale representation theorem (or since the market is complete), there exists α ˆ ∈ A s.t. ˆ t,x α ˆs X ˆ st,x = X ˆ st,x σ α dX ˆ s dWsQ = s dSs , Ss ˆ t,x is a wealth process controlled by the proportion α which means that X ˆ , and starting from initial capital x at time t. ˆ t,x is the I From the representation (14) of the value function, this proves that X optimal wealth process: h i t,x ˆ v(t, x) = E U (XT ) .
2. Merton portfolio/consumption choice on infinite horizon In addition to the investment α in the stock, the agent can also consume from his wealth: → (ct )t≥0 consumption per unit of wealth I The wealth process, controlled by (α, c) is governed by: dXt = Xt (r + αt (µ − r) − ct ) dt + Xt αt σdWt .
• The preferences of the agent is described by a utility U from consumption, and the goal is to maximize over portfolio/consumption the expected utility from intertemporal consumption up to a random time horizon: Z τ −βt x v(x) = sup E e U (ct Xt )dt , (α,c)
x > 0.
0
We assume that τ is independent of F∞ (market information), and follows E(λ). I Infinite horizon stochastic control problem: Z ∞ −(β+λ)t x v(x) = sup E e U (ct Xt )dt , (α,c)
0
x > 0.
HJB equation
1 (β + λ)v − rxv 0 − sup[a(µ − r)v 0 + a2 x2 σ 2 v 00 ] − sup[U (cx) − cxv 0 ] = 0, 2 c≥0 a∈A • Explicit solution for CRRA utility function: U (x) = xp /p. Under the condition that β + λ > ρ, we have v(x) = K U (x),
with K =
1−p β+λ−ρ
1−p
The optimal portfolio/consumption strategies are: 1 a ˆ = arg max[a(µ − r) − a2 (1 − p)σ 2 ] a∈A 2 1 1 1 cˆ = (v 0 (x)) p−1 = K p−1 . x
.
x > 0.
VII. Some other classes of stochastic control problems • Ergodic and risk-sensitive control problems - Risk-sensitive control problem: Z T 1 ln E exp θ f (Xt , αt )dt lim sup T →∞ θT 0 → Applications in finance: Bielecki, Pliska, Fleming, Sheu, Nagai, Davis, etc ... - Large deviations control problem: 1 XT lim sup P ≥x T T →∞ T ←→ Dual of risk-sensitive control problem → Applications in finance: Pham, Sekine, Nagai, Hata, Sheu
• Optimal stopping problems: The control decision is a stopping time τ where we decide to stop the process → Value function of optimal stopping problem (over a finite horizon): Z τ t,x t,x v(t, x) = sup E f (Xs )ds + g(Xτ ) . t≤τ ≤T
t
I The HJB equation is a free boundary or variational inequality: ∂v min − − Lv − f , v − g = 0, ∂t where L is the infinitesimal generator of the Markov process X. → Typical applications in finance in American option pricing
• Impulse and optimal switching problems: The control is a sequence of increasing stopping times (τn )n associated to a sequence of actions (ζn )n : τn represents the time decision when we decide to intervene on the state system X by using an action ζn Fτn -measurable: Xτn− → Γ(Xτn− , ζn ) → Value function: "Z
T
v(t, x) = sup E (τn ζn )
t
# f (Xst,x )ds + g(XTt,x ) +
X
c(Xτn− , ζn ) .
n
I The HJB equation is a quasi-variational inequality: ∂v − Lv − f , v − Hv = 0, min − ∂t where L is the infinitesimal generator of the Markov process X, and H is a nonlocal operator associated to the jump and cost induced by an action: Hv(t, x) = sup v(t, Γ(x, e)) + c(x, e) . e∈E
→ Various applications in finance: • Transaction costs and liquidity risk models, where trading times take place discretely • Real options and firm investment problems, where decisions represent change of regimes or production technologies
Lecture 2 : Viscosity solutions and stochastic control
• Non smoothness of value functions: a motivating financial example • Introduction to viscosity solutions • Viscosity properties of the dynamic programming equation • Comparison principles • Application: Super-replication in uncertain volatility models
I. Non smoothness of value functions: a motivating financial example • Consider the controlled diffusion process dXs = αs Xs dWs , with an unbounded control α valued in A = R+ : Uncertain volatility model. Consider the stochastic control problem v(t, x) = sup E[g(XTt,x )], (t, x) ∈ [0, T ] × (0, ∞), α∈A
←→ Superreplication cost of an option payoff g(XT ). I If v were smooth, it would be a classical solution to the HJB equation: i h1 2 2 (t, x) ∈ [0, T ) × (0, ∞). vt + sup a x vxx = 0, a∈R+ 2
(1)
But, for the supremum in a ∈ R to be finite and HJB equation (1) to be well-posed, we must have vxx ≤ 0,
i.e. v(t, .) is concave in x, for any t ∈ [0, T ).
• Now, by taking the zero control in the definition of v, we get v(t, x) ≥ g(x), which combined with the concavity of v(t, .), implies: v(t, x) ≥ gˆ(x),
t < T,
where gˆ is the concave envelope of g: the smallest concave function above g. • Moreover, since g ≤ gˆ, and by Jensen’s inequality and martingale property of X, we have v(t, x) ≤ sup E[ˆ g (XTt,x )] α∈A ≤ sup gˆ E[XTt,x ] = gˆ(x). α∈A
I Therefore, v(t, x) = gˆ(x),
∀(t, x) ∈ [0, T ) × (0, ∞).
→ There is a contradiction with the smoothness of v, whenever gˆ is not smooth!, for example when g is concave (hence equal to gˆ) but not smooth. • Need to consider the case where the supremum in HJB can explode (singular case) and to define weak solutions for HJB equation → Notion of viscosity solutions (Crandall, Ishii, P.L. Lions)
II. Introduction to viscosity solutions Consider nonlinear parabolic second-order partial differential equations: F (t, x, w,
∂w 2 , Dx w, Dxx w) = 0, ∂t
(t, x) ∈ [0, T ) × O,
(2)
where O is an open subset of Rn and F is a continuous function of its arguments, satisfying the ellipticity condition: for all (t, x) ∈ [0, T ) × O, r ∈ R, (q, p) ∈ R × Rn , c ∈ Sn , M, M c =⇒ F (t, x, r, q, p, M ) ≥ F (t, x, r, q, p, M c), M ≤M
(3)
and the parabolicity condition: for all t ∈ [0, T ), x ∈ O, r ∈ R, q, qˆ ∈ R, p ∈ Rn , M ∈ Sn , q ≤ qˆ =⇒ F (t, x, r, q, p, M ) ≥ F (t, x, r, qˆ, p, M ).
• Typical example: HJB equation F (t, x, r, q, p, M ) = −q − H(x, p, M ), where H is the Hamiltonian function of the stochastic control problem: 1 H(x, p, M ) = sup b(x, a).p + tr (σσ 0 (x, a)M ) + f (x, a) 2 a∈A
(4)
Intuition for the notion of viscosity solutions • Assume that w is a smooth supersolution to (2). Let ϕ be a smooth test function on [0, T ) × O, and (t¯, x¯) ∈ [0, T ) × O be a minimum point of w − ϕ: 0 = (w − ϕ)(t¯, x¯) = min(w − ϕ). In this case, the first and second-order optimality conditions imply ∂(w − ϕ) ¯ (t, x¯) ≥ 0 (= 0 if t¯ > 0) ∂t Dx w(t¯, x¯) = Dx ϕ(t¯, x¯) and Dx2 w(t¯, x¯) ≥ Dx2 ϕ(t¯, x¯). I From the ellipticity and parabolicity conditions (3) and (4), we deduce that ∂ϕ ¯ (t, x¯), Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) ∂t ∂w (t¯, x¯), Dx w(t¯, x¯), Dx2 w(t¯, x¯)) ≥ 0, ≥ F (t¯, x¯, w(t¯, x¯), ∂t F (t¯, x¯, ϕ(t¯, x¯),
• Similarly, if w is a classical subsolution to (2), then for all test functions ϕ, and (t¯, x¯) ∈ [0, T ) × O such that (t¯, x¯) is a maximum point of w − ϕ, we have ∂ϕ ¯ F (t¯, x¯, ϕ(t¯, x¯), (t, x¯), Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) ≤ 0. ∂t
General definition of (discontinuous) viscosity solutions Given a locally bounded function w on [0, T ] × O, we define its upper-semicontinuous (usc) envelope w∗ and lower-semicontinuous (lsc) envelope w∗ by w∗ (t, x) = lim sup w(t0 , x0 ),
w∗ (t, x) = lim inf w(t0 , x0 ).
t0
t0
Remark. w∗ ≤ w ≤ w∗ , and w is usc (resp. lsc) on [0, T ) × O iff w = w∗ (resp. = w∗ ), and w is continuous [0, T ) × O iff w = w∗ = w∗ .
Definition .1 Let w : [0, T ] × O → R be locally bounded. (i) w is a viscosity supersolution (resp. subsolution) of (2) on [0, T ) × O if ∂ϕ ¯ 2 F (t¯, x¯, ϕ(t¯, x¯), (t, x¯), Dx ϕ(t¯, x¯), Dxx ϕ(t¯, x¯)) ≥ (resp ≤) 0, ∂t for all (t¯, x¯) ∈ [0, T ) × O, and test functions ϕ such that (t¯, x¯) is a minimum (resp. maximum) point of w∗ − ϕ (resp. w∗ − ϕ). (ii) w is a viscosity solution of (2) on [0, T ) × O if it is both a subsolution and supersolution of (2).
III. Viscosity properties of the DPE We turn back to the stochastic control problem: Z T t,x v(t, x) = sup E f (Xst,x , αs )ds + g(XT ) , (t, x) ∈ [0, T ] × Rn , α∈A
t
with Hamiltonian function H on Rn × Rn × Sn : 1 H(x, p, M ) = sup b(x, a).p + tr (σσ 0 (x, a)M ) + f (x, a) . 2 a∈A I We introduce the domain of H as dom(H) = {(x, p, M ) ∈ Rn × Rn × Sn : H(x, p, M ) < ∞} , and make the following hypothesis (DH): H is continuous on int(dom(H)) and there exists a continuous function G on Rn × Rn × Sn such that (x, p, M ) ∈ dom(H) ⇐⇒ G(x, p, M ) ≥ 0. Example. In the example considered at the beginning of this lecture: i h1 2 2 H(x, p, M ) = sup a x M , a∈R 2 and so G(x, p, M ) = −M.
• Viscosity property inside the domain
Theorem .1 The value function v is a viscosity solution to the HJB variational inequality ∂v min − − H(x, Dx v, Dx2 v) , G(x, Dx v, Dx2 v) = 0, ∂t
on [0, T ) × Rn .
Remark. In the regular case when the Hamiltonian H is finite on the whole domain Rn × Rn × Sn (this occurs typically when the control space is compact), the condition (DH) is satisfied with any choice of strictly positive continuous function G. In this case, the HJB variational inequality is reduced to the regular HJB equation: −
∂v (t, x) − H(x, Dx v, Dx2 v) = 0, ∂t
(t, x) ∈ [0, T ) × Rn ,
which the value function satisfies in the viscosity sense. Hence, the above Theorem states a general viscosity property including both the regular and singular case.
Proof of viscosity supersolution property • Let (t¯, x¯) ∈ [0, T ) × Rn and let ϕ ∈ C 2 ([0, T ) × Rn ) be a test function such that 0 = (v∗ − ϕ)(t¯, x¯) =
min (v∗ − ϕ).
[0,T )×Rn
(5)
By definition of v∗ (t¯, x¯), there exists a sequence (tm , xm )m in [0, T ) × Rn such that (tm , xm ) → (t¯, x¯) and v(tm , xm ) → v∗ (t¯, x¯), when m goes to infinity. By the continuity of ϕ and by (5) we also have that γm := v(tm , xm ) − ϕ(tm , xm ) → 0. • Let α ∈ A, a constant process equal to a ∈ A, and Xstm ,xm the associated controlled process. Let τm = inf{s ≥ tm : |Xstm ,xm − xm | ≥ η}, with η > 0 a fixed constant. Let (hm ) be a strictly positive sequence such that hm → 0 and
γm → 0. hm
We apply the first part of the DPP for v(tm , xm ) to θm := τm ∧ (tm + hm ) and get Z θm t ,x v(tm , xm ) ≥ E f (s, Xstm ,xm , a)ds + v(θm , Xθmm m ) . tm
Equation (5) implies that v ≥ v∗ ≥ ϕ, thus Z θ m ϕ(tm , xm ) + γm ≥ E f (s, Xstm ,xm , a)ds + ϕ(θm , Xθtmm ,xm ) . tm
Apply Itˆo’s formula to ϕ(s, Xstm ,xm ) between tm and θm : Z θm γm 1 ∂ϕ a tm ,xm − L ϕ − f (s, Xs , a)ds ≥ 0. + E − hm hm tm ∂t
(6)
Now, send m to infinity: by the mean value theorem, and the dominated convergence theorem, we get −
∂ϕ ¯ (t, x¯) − La ϕ(t¯, x¯) − f (t¯, x¯, a) ≥ 0. ∂t
Since a is arbitrary in A, and by definition of H, this means: ∂ϕ ¯ (t, x¯) − H(¯ x, Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) ≥ 0. ∂t In particular, (¯ x, Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) ∈ dom(H), and so −
G(¯ x, Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) ≥ 0. Therefore, i h ∂ϕ 2 2 ¯ ¯ ¯ ¯ ¯ (t, x¯) − H(¯ x, Dx ϕ(t, x¯), Dx ϕ(t, x¯)) , G(¯ x, Dx ϕ(t, x¯), Dx ϕ(t, x¯)) ≥ 0, min − ∂t which is the required supersolution property.
Proof of viscosity subsolution property • Let (t¯, x¯) ∈ [0, T ) × Rn and let ϕ ∈ C 2 ([0, T ) × Rn ) be a test function such that 0 = (v ∗ − ϕ)(t¯, x¯) =
max n (v ∗ − ϕ).
(7)
[0,T )×R
As before, there exists a sequence (tm , xm )m in [0, T ) × Rn s.t. (tm , xm ) → (t¯, x¯) and v(tm , xm ) → v ∗ (t¯, x¯), γm := v(tm , xm ) − ϕ(tm , xm ) → 0. • We will show the result by contradiction, and assume on the contrary that ∂ϕ ¯ (t, x¯) − H(¯ x, Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) > 0, ∂t G(¯ x, Dx ϕ(t¯, x¯), Dx2 ϕ(t¯, x¯)) > 0.
− and
Under (DH), there exists η > 0 such that −
∂ϕ (t, y) − H(y, Dx ϕ(t, y), Dx2 ϕ(t, y)) > 0, ∂t
for (t, x) ∈ B(t¯, η) × B(¯ x, η). (8)
Observe that we can assume w.l.o.g. in (7) that (t¯, x¯) achieves a strict maximum so that max (v ∗ − ϕ) =: −δ < 0,
∂p B((t¯,¯ x),η)
where ∂p B((t¯, x¯), η) = [t¯, t¯ + η] × ∂B(¯ x, η) ∪ {t¯ + η} × B(¯ x, η).
(9)
I We apply the second part of DP: there exists α ˆ m ∈ A s.t. Z θ m δ ˆ stm ,xm , α ˆ tm ,xm ) , v(tm , xm ) − ≤ E f (X ˆ sm )ds + v(θm , X θm 2 tm
(10)
ˆ stm ,xm ) ∈ where θm = inf{s ≥ tm : (s, X / B(t¯, η) × B(¯ x, η)}. Observe by continuity of ˆ tm ,xm ) ∈ ∂p B((t¯, x¯), η) so that from (9)-(10): the state process that (θm , X θm Z θm δ t ,x t ,x m m m ˆ sm m , α ˆ ≤ E f (X ϕ(tm , xm ) + γm − ˆ s )ds + ϕ(θm , X ) − δ. θm 2 tm ˆ stm ,xm ) between tm and θm , we then get after noting that Apply Itˆo’s formula to ϕ(s, X the stochastic integral vanishes in expectation: Z θ m ∂ϕ δ m ˆ stm ,xm , α − Lαˆs ϕ − f (s, X − ˆ sm )ds ≤ −δ. γm − + E 2 ∂t tm
(11)
Now, from (8) and definition of H, we have ∂ϕ ˆ stm ,xm ) − Lαˆsm ϕ(s, X ˆ stm ,xm ) − f (X ˆ stm ,xm , α (s, X ˆ sm ) ∂t ∂ϕ ˆ stm ,xm ) − H(s, Dx ϕ(s, X ˆ stm ,xm ), Dx2 ϕ(s, X ˆ stm ,xm )) ≥ − (s, X ∂t > 0, for tm ≤ s ≤ θm . −
Plugging into (11), this implies γm −
δ ≤ −δ, 2
and we get the contradiction by sending m to infinity: −δ/2 ≤ −δ.
(12)
• Terminal condition Due to the singularity of the Hamiltonian H, the value function may be discontinuous at T , i.e. v(T − , x) may be different from g(x). The right terminal condition is given by the relaxed terminal condition: Theorem .2 The value function v is a viscosity solution to min v − g , G(x, Dx v, Dx2 v) = 0,
on {T } × Rn .
(13)
This means that v∗ (T, .) is a viscosity supersolution to min v∗ (T, x) − g(x) , G(x, Dx v∗ (T, x), Dx2 v∗ (T, x)) ≥ 0,
on Rn .
(14)
on Rn .
(15)
and v ∗ (T, .) is a viscosity subsolution to min v ∗ (T, x) − g(x) , G(x, Dx v ∗ (T, x), Dx2 v ∗ (T, x)) ≤ 0,
Remark. Denote by gˆ the upper G-envelope of g, defined as the smallest function above g, and viscosity supersolution to G(x, Dˆ g , D2 gˆ) ≥ 0. Then v∗ (T, x) ≥ gˆ(x). On the other hand, since gˆ is a viscosity supersolution to (14), and if a comparison principle holds for (13), then v ∗ (T, x) ≤ gˆ(x). This implies v(T − , x) = v∗ (T, x) = v∗ (T, x) = gˆ(x). In the regular case, we have gˆ = g, and v is continuous at T .
IV. Strong comparison principles and uniqueness Consider the DPE satisfied by the value function ∂v 2 2 min − − H(x, Dx v, Dx v) , G(x, Dx v, Dx v) = 0, ∂t min v(T, x) − g(x) , G(x, Dx v, Dx2 v) = 0,
on [0, T ) × Rn . (16) on {T } × Rn .
(17)
• We say that a strong comparison principle holds for (16)-(17) when the following statement is true: If u is an usc viscosity subsolution to (16)-(17) and w is a lsc viscosity supersolution to (16)-(17), satisfying some growth condition, then u ≤ w. Remark. The arguments for proving comparison principles are: - dedoubling variables technique - Ishii’s Lemma → Standard reference: user’s guide of Crandall, Ishii’s Lions (92).
Consequence of strong comparison principles • Uniqueness and continuity Suppose that v and w are two viscosity solutions to (16)-(17). This means that v ∗ is a viscosity subsolution to (16)-(17), and w∗ is a viscosity supersolution to (16)-(17), and vice-versa. By the strong comparison principle, we get: v ∗ ≤ w∗
and
w∗ ≤ v∗ .
Since w∗ ≤ w∗ , v∗ ≤ v ∗ , this implies: v ∗ = v∗ = w∗ = w∗ . Therefore, v = w, v∗ = v ∗ ,
i.e. uniqueness i.e. continuity of v on [0, T ) × Rn .
• Conclusion The value function of the stochastic control problem is the unique continuous viscosity solution to (16)-(17) (satisfying some growth condition).
V. Application: superreplication in uncertain volatility model Consider the controlled diffusion dXs = αs Xs dWs ,
t ≤ s ≤ T,
with the control process α valued in A = [a, a ¯], where 0 ≤ a ≤ a ¯ ≤ ∞. Given a continuous function g on R+ , we consider the stochastic control problem: v(t, x) = sup E g(XTt,x ) , (t, x) ∈ [0, T ] × (0, ∞). α∈A
Financial interpretation α represents the uncertain volatility process of the stock price X, and the function g represents the payoff of an European option of maturity T . The value function v is the superreplication cost for this option, that is the minimum capital required to superhedge (by means of trading strategies on the stock) the option payoff at maturity T whatever the realization of the uncertain volatility.
The Hamiltonian of this stochastic control problem is H(x, M ) =
1 2 2 a x M , (x, M ) ∈ (0, ∞) × R. 2 a∈[a,¯ a] sup
→ We shall then distinguish two cases: a ¯ finite or not. • Bounded volatility: a ¯ < ∞. In this regular case, H is finite on the whole domain (0, ∞) × R, and is given by H(x, M ) =
1 2 a ˆ (M )x2 M, 2
with ( a ˆ(M ) =
a ¯
if M ≥ 0
a
if M < 0.
I v is continuous on [0, T ] × (0, ∞), and is the unique viscosity solution with linear growth condition to the so-called Black-Scholes-Barenblatt equation 1 2 ˆ (vxx ) x2 vxx = 0, (t, x) ∈ [0, T ) × (0, ∞), vt + a 2 satisfying the terminal condition v(T, x) = g(x),
x ∈ (0, ∞).
Remark. If g is convex, then v is equal to the Black-Scholes price with volatility a ¯, which is convex in x, so that a ˆ(vxx ) = a ¯.
• Unbounded volatility: a ¯ = ∞. In this singular case, the Hamiltonian is given by ( 1 2 2 if G(M ) := −M ≥ 0 2a x M H(x, M ) = ∞ if − M < 0. I v is the unique viscosity solution to the HJB variational inequality 1 min − vt − a2 x2 vxx , −vxx = 0, 2 min v − g , −vxx = 0,
on [0, T ) × (0, ∞),
(18)
on {T } × (0, ∞).
(19)
Explicit solution to (18)-(19) Denote by gˆ the concave envelope of g, i.e. the solution to min[ˆ g − g , −ˆ gxx ] = 0. Let us consider the Black-Scholes price with volatility a of the option payoff gˆ, i.e. i h t,x ˆ w(t, x) = E gˆ XT , where ˆ s = aX ˆ s dWs , t ≤ s ≤ T, X ˆ t = x. dX Then, v = w,
on [0, T ) × (0, ∞).
Proof. Indeed, the function w is solution to the Black-Scholes equation: 1 wt + a2 x2 wxx = 0, on [0, T ) × (0, ∞) 2 w(T, x) = gˆ(x), x ∈ (0, ∞). Moreover, w inherits from gˆ the concavity property, and so −wxx ≥ 0, (t, x) ∈ [0, T ) × (0, ∞). (This holds true in the viscosity sense) I Therefore, w satisfies the same HJB variational inequality as v: 1 min − wt − a2 x2 wxx , −wxx = 0, 2 min w − g , −wxx = 0,
on [0, T ) × (0, ∞), on {T } × (0, ∞).
We conclude by uniqueness result. Remark. When a = 0, we have w = gˆ, and so v(t, x) = gˆ(x) on [0, T ) × (0, ∞).
Lecture 3 : Backward Stochastic Differential Equations and stochastic control
• Introduction • General properties of BSDE • The Markov case : nonlinear Feynman-Kac formula. Simulation of BSDE • Application: CRRA utility maximization • Reflected BSDE and optimal stopping problem • BSDE with constrained jumps and quasi-variational inequalities
I. Introduction • BSDEs first introduced by Bismut (73): adjoint equation in Pontryagin maximum principle (linear BSDEs) • Emergence of the theory since the seminal paper by Pardoux and Peng (90): general BSDEs • BSDEs widely used in stochastic control and mathematical finance • Hedging and pricing problems ↔ linear and reflected BSDE • Portfolio optimization, risk measure ↔ nonlinear BSDE, reflected and constrained BSDEs • Improve existence and uniqueness of BSDEs, especially quadratic BSDEs • BSDE provide a probabilistic representation of nonlinear PDEs: nonlinear FeynmanKac formulae → Numerical methods for nonlinear PDEs
II. General results on BSDEs Let W = (Wt )0≤t≤T be a standard d-dimensional Brownian motion on (Ω, F, F, P ) where F = (Ft )0≤t≤T is the natural filtration of W , and T is a fixed finite horizon. Notations • P: set of progressively measurable processes on Ω × [0, T ] • S2 (0, T ): set of elements Y ∈ P such that E sup |Yt |2 < ∞, 0≤t≤T
• H2 (0, T )d : set of elements Z ∈ P, Rd -valued, such that Z T 2 E |Zt | dt < ∞. 0
Definition of BSDE A (one-dimensional) Backward Stochastic Differential Equation (BSDE in short) is written in differential form as −dYt = f (t, Yt , Zt )dt − Zt .dWt ,
YT = ξ,
(1)
where the data is a pair (ξ, f ), called terminal condition and generator (or driver): ξ ∈ L2 (Ω, FT , P) , f (t, ω, y, z) is P ⊗ B(R × Rd )-measurable. A solution to (1) is a pair (Y, Z) ∈ S2 (0, T ) × H2 (0, T )d such that Z T Z T Yt = ξ + f (s, Ys , Zs )ds − Zs .dWs , 0 ≤ t ≤ T. t
t
Under some specific assumptions on the generator f , there is existence and uniqueness of a solution to the BSDE (1). Standard Lipschitz assumption (H1) • f is uniformly Lipschitz in (y, z), i.e. there exists a positive constant C s.t. for all (y, z, y 0 , z 0 ): |f (t, y, z) − f (t, y , z )| ≤ C |y − y | + |z − z | , 0
0
0
0
dt ⊗ dP a.e.
• The process {f (t, 0, 0), t ∈ [0, T ]} ∈ H2 (0, T ) Theorem (Pardoux and Peng 90) Under (H1), there exists a unique solution (Y, Z) to the BSDE (1). Proof. (a) Assume first the case where f does not depend on (y, z), and consider the martingale
Z
Mt = E ξ + 0
T
f (t, ω)dt Ft ,
which is square-integrable under (H1), i.e. M ∈ S2 (0, T ). By the martingale representation theorem, there exists a unique Z ∈ H2 (0, T )d s.t. Z t Mt = M0 + Zs .dWs , 0 ≤ t ≤ T. 0
Then, the process Z h Yt := E ξ + t
T
Z t i f (s, ω)ds Ft = Mt − f (s, ω)ds, 0 ≤ t ≤ T,
satisfies (with Z) the BSDE (1).
0
Proof. (b) Consider now the general Lipschitz case. As in the deterministic case, we give a proof based on a fixed point method. Let us consider the function Φ on S2 (0, T )m × H2 (0, T )d , mapping (U, V ) ∈ S2 (0, T ) × H2 (0, T )d to (Y, Z) = Φ(U, V ) defined by Z
T
Z f (s, Us , Vs )ds −
Yt = ξ + t
T
Zs .dWs . t
This pair (Y, Z) exists from Step (a). We then see that (Y, Z) is a solution to the BSDE (1) if and only if it is a fixed point of Φ. Let (U, V ), (U 0 , V 0 ) ∈ S2 (0, T )×H2 (0, T )d and (Y, Z) = Φ(U, V ), (Y 0 , Z 0 ) = Φ(U 0 , V 0 ). ¯ = (Y − Y 0 , Z − Z 0 ) and f¯t = f (t, Ut , Vt ) − We set (U¯ , V¯ ) = (U − U 0 , V − V 0 ), (Y¯ , Z) f (t, Ut0 , Vt0 ). Take some β > 0 to be chosen later, and apply Itˆo’s formula to eβs |Y¯s |2 between s = 0 and s = T : Z
T
eβs β|Y¯s |2 − 2Y¯s .f¯s ds Z0 T Z T βs ¯ 2 − e |Zs | ds − 2 eβs Y¯s0 Z¯s .dWs .
|Y¯0 |2 = −
0
0
By taking the expectation, we get hZ T i hZ T i 2 βs 2 2 βs ¯ ¯ ¯ ¯ ¯ E|Y0 | + E e β|Ys | + |Zs | ds = 2E e Ys .fs ds 0 0 hZ T i eβs |Y¯s |(|U¯s | + |V¯s |)ds ≤ 2Cf E 0 hZ T i hZ T i 1 2 βs ¯ 2 βs ¯ 2 2 ¯ ≤ 4Cf E e |Ys | ds + E e (|Us | + |Vs | )ds 2 0 0
Proof continued. (b) Now, we choose β = 1 + 4Cf2 , and obtain hZ T hZ T i i 1 βs 2 2 βs 2 2 E e |Y¯s | + |Z¯s | ds ≤ E e (|U¯s | + |V¯s | )ds . 2 0 0 This shows that Φ is a strict contraction on the Banach space S2 (0, T ) × H2 (0, T )d endowed with the norm k(Y, Z)kβ
hZ = E
T
e
βs
i 12 . |Ys | + |Zs | ds 2
2
0
We conclude that Φ admits a unique fixed point, which is the solution to the BSDE (1).
Non-Lipschitz conditions on the generator • f is continuous in (y, z) and satisfies a linear growth condition on (y, z). Then, there exists a minimal solution to the BSDE (1). (Lepeltier and San Martin 97) • f is continuous in (y, z), linear in y, and quadratic in z, and ξ is bounded. Then, there exists a unique bounded solution to the BSDE (1) (Kobylanski 00).
III. The Markov case: non-linear Feynman-Kac formulae Linear Feynman-Kac formula Consider the linear parabolic PDE ∂v (t, x) + Lv(t, x) + f (t, x) = 0, ∂t v(T, .) = g,
on [0, T ) × Rd
(2)
on Rd ,
(3)
where L is the second-order differential operator 1 Lv = b(x).Dx v + tr(σσ 0 (x)Dx2 v). 2 I Consider the (forward) diffusion process dXt = b(Xt )dt + σ(Xt )dWt . Then, by Itˆo’s formula to v(t, Xt ) between t and T , with v smooth solution to (2)-(3): Z T Z T v(t, Xt ) = g(XT ) + f (s, Xs )ds − Dx v(s, Xs )0 σ(Xs )dWs . t
t
It follows that the pair (Yt , Zt ) = (v(t, Xt ), σ 0 (Xt )Dx v(t, Xt )) solves the linear BSDE: Z T Z T Yt = g(XT ) + f (s, Xs )ds − Zs dWs . t
t
Remark We can compute the solution v(0, X0 ) = Y0 by the Monte-Carlo expectation: Z T Y0 = E g(XT ) + f (s, Xs )ds . 0
Non linear Feynman-Kac formula Consider the semilinear parabolic PDE ∂v + Lv + f (t, x, v, σ 0 Dx v) = 0, ∂t v(T, .) = g,
on [0, T ) × Rd
(4)
on Rd ,
(5)
I The corresponding BSDE is Z
T
Z
T
f (s, Xs , Ys , Zs )ds −
Yt = g(XT ) + t
Zs dWs ,
(6)
t
in the sense that: • the pair (Yt , Zt ) = (v(t, Xt ), σ 0 (Xt )Dx v(t, Xt )) solves (6) • Conversely, if (Y, Z) is a solution to (6), then Yt = v(t, Xt ) for some deterministic function v, which is a viscosity solution to (4)-(5).
I The time discretization and simulation of the BSDE (6) provides a numerical method for solving the semilinear PDE (4)-(5)
Simulation of BSDE: time discretization • Time grid π = (ti ) on [0, T ] : ti = i∆t, i = 0, . . . , N , ∆t = T /N • Forward Euler scheme X π for X : starting from Xtπ0 = x, Xtπi+1 := Xtπi + b(Xtπi )∆t + σ(Xtπi ) Wti+1 − Wti
• Backward Euler scheme (Y π , Z π ) for (Y, Z) : starting from YtπN = g(XtπN ), Ytπi = Ytπi+1 + f (Xtπi , Ytπi+1 , Ztπi )∆t − Ztπi . Wti+1 − Wti
and take conditional expectation: i h π π π π π Yti = E Yti+1 + f (Xti , Yti+1 , Zti )∆t Xtπi To get the Z-component, multiply (7) by Wti+1 − Wti and take expectation: i 1 h π π E Yti+1 (Wti+1 − Wti ) Xtπi Zti = ∆t
(7)
Simulation of BSDE: numerical methods How to compute these conditional expectations! several approaches: • Regression based algorithms (Longstaff, Schwartz) Choose q deterministic basis functions ψ1 , . . . , ψq , and approximate Ztπi
q i h X π π = E Yti+1 (Wti+1 − Wti ) Xti ' αk ψk (Xtπi ) k=1
where α = (αk ) solve the least-square regression problem: q h i2 X π π ¯ Y (Wt − Wt ) − arg infq E α ψ (X ) k k ti+1 ti i+1 i α∈R
k=1
Here E¯ is the empirical mean based on Monte-Carlo simulations of Xtπi , Xtπi+1 , Wti+1 − Wt i . → Efficiency enhanced by using the same set of simulation paths to compute all conditional expectations. • Other methods: • Malliavin Monte-Carlo approach (P.L. Lions, Regnier) • Quantization methods (Pag`es) → Important literature: Kohatsu-Higa, Pettersson (01), Ma, Zhang (02), Bally and Pag`es (03), Bouchard, Ekeland, Touzi (04), Gobet et al. (05), Soner and Touzi (05), Peng, Xu (06), Delarue, Menozzi (07), Bender and Zhang (08), etc ...
IV. Application: CRRA utility maximization • Consider a financial market model with one riskless asset S 0 = 1, and n stocks of price process dSt = diag(St ) µt dt + σt dWt , wher W is a d-dimensional Brownian motion (with d ≥ n), b, σ bounded adapted processes, σ of full rank n. Consider an agent investing in the stocks a fraction α of his wealth X at any time: dXt = Xt αt0 diag(St )−1 dSt = Xt (αt0 µt dt + αt0 σt dWt )
(8)
A0 : set of F-adapted processes α valued in A closed convex set of Rn , and satisfying: RT 0 2 RT 0 µ |dt + |α t t 0 |αt σt | dt < ∞, → (8) is well-defined. 0 • Given a utility function U on (0, ∞), and starting from initial capital X0 > 0, the objective of the agent is: V0 := sup E[U (XTα )].
(9)
α∈A
Here, X α is the solution to (8) controlled by α ∈ A0 , and starting from X0 at time 0, and A is the subset of elements α ∈ A0 s.t. {U (Xτα ), τ ∈ T0,T } is uniformly integrable. I We solve (9) by dynamic programming and BSDE.
• Value function processes: For t ∈ [0, T ], and α ∈ A, we denote by: At (α) = {β ∈ A : β.∧t = α.∧t } , and define the family of F-adapted processes h i β Vt (α) := ess sup E U (XT ) Ft , 0 ≤ t ≤ T. β∈At (α)
• Dynamic programming (DP) • For any α ∈ A, the process {Vt (α), 0 ≤ t ≤ T } is a supermartingale • There exists an optimal control α ˆ ∈ A to V0 if and only if the martingale property holds, i.e. the process {Vt (ˆ α), 0 ≤ t ≤ T } is a martingale. I In the sequel, we exploit the DP in the case of CRRA utility functions: U (x) = xp /p, p < 1. The key observation is the property that the F-adapted process Yt :=
Vt (α) > 0 U (Xtα )
does not depend on α ∈ A,
and YT = 1.
I We adopt a BSDE verification approach: we are looking for (Y, Z) solution to Z T Z T Yt = 1 + f (s, ω, Ys , Zs )ds − Zs dWs , (10) t
t
for some generator f to be determined such that • For any α ∈ A, the process {U (Xtα )Yt , 0 ≤ t ≤ T } is a supermartingale • There exists α ˆ ∈ A for which {U (Xtαˆ )Yt , 0 ≤ t ≤ T } is a martingale.
I By applying Itˆo’s formula to U (Xtα )Yt , the supermartingale property for all α ∈ A, and the martingale property for some α ˆ imply that f should be equal to h i 1−p f (t, Yt , Zt ) = p sup (µt Yt + σt Zt ).a − Yt |σt a|2 , 2 a∈A with a candidate for the optimal control given by h i 1−p 2 Yt |σt a| , 0 ≤ t ≤ T. α ˆ t ∈ arg max (µt Yt + σt Zt ).a − 2 a∈A
(11)
(12)
• Existence and uniqueness of a solution to the BSDE (10)-(11): • Change of variables Y˜ = ln Y , Z˜ = Z/Y ˜ satisfy a quadratic BSDE. Then, we rely on results by Kobylanski (00) • (Y˜ , Z) • Existence and uniqueness of (Y, Z) ∈ S∞ (0, T ) ∩ H2 (0, T )d • Verification argument: let (Y, Z) be the solution to (10)-(11) • By construction U (Xtα )Yt is a (local)-supermartingale + integrability conditions on α ∈ A: → it is a supermartingale → supα∈A E[U (XTα )] ≤ U (X0 )Y0 . • By BMO techniques, we show that α ˆ defined in (12) lies in A → U (Xtαˆ )Yt is a martingale → E[U (XTαˆ )] = U (X0 )Y0 • We conclude that V0 := supα∈A E[U (XTα )] = U (X0 )Y0 , and α ˆ is an optimal control.
Markov cases • Merton model: the coefficients of the stock price µ(t) and σ(t) are deterministic In this deterministic case, the BSDE (10)-(11) is reduced to an ODE: Z T h i 1−p 2 |σ(t)a| =: yρ(t) Y (t) = 1 + f (s, Y (s))ds, f (t, y) = y p sup µ(t).a − 2 a∈A t RT
and the solution is given by: Y (t) = e
t
ρ(s)ds
→ we find again the solution to the
Merton problem: Z V0 = U (X0 ) exp 0
T
ρ(s)ds .
• Factor model: the coefficients of the stock price µ(t, Lt ) and σ(t, Lt ) depend on a factor process dLt = η(Lt )dt + dWt . ˜ = (ln Y, Z/Y ) is written as: In this case, the BSDE for (Y˜ , Z) Z T Z T Yt = f˜(s, Ls , Y˜s , Z˜s )ds − Z˜s dWs t
t
with a quadratic (in z) generator h i 1 1−p 2 ˜ f (t, `, y, z) = p sup (µ(t, `) + σ(t, `)z).a − |σ(t, `)a| + z 2 . 2 2 a∈A → Y˜t = ϕ(t, Lt ), with a corresponding semilinear PDE for ϕ: ∂ϕ 1 + η(`).D` ϕ + tr(D`2 ϕ) + f˜(t, `, ϕ, D` ϕ) = 0, ∂t 2
ϕ(T, `) = 0.
→ Value function: V0 = U (X0 ) exp ϕ(0, L0 ) .
Remark The BSDE approach and dynamic programming is also well-suitable for exponential utility maximization: → Many papers: El Karoui, Rouge (00), Hu, Imkeller, Muller (04), Sekine (06), Becherer (06), etc ...
V. Reflected BSDE and optimal stopping problem We consider a class of BSDEs where the solution Y is constrained to stay above a given process L, called obstacle. An increasing process K is introduced for pushing the solution upwards, above the obstacle → Notion of reflected BSDE: • Given pair of terminal condition/generator (ξ, f ) and a continuous obstacle process (Lt ) s.t. ξ ≥ LT , find a triple of adapted processes (Y, Z, K) with K nondecreasing s.t. T
Z
Z f (s, Ys , Zs )ds −
Yt = ξ + t
T
Zs dWs + KT − Kt , 0 ≤ t ≤ T,
(13)
t
Yt ≥ Lt , 0 ≤ t ≤ T, Z T (Yt − Lt )dKt = 0.
(14) (15)
0
• Connection with optimal stopping problem: in the case where f (t, ω) does not depend on (y, z), there exists a unique solution to (13)-(14)-(15) given by i hZ τ (16) Yt = ess sup E f (s)ds + Lτ 1τ
t
Arguments of proof. Rt • Snell envelope of Ht := 0 f (s)ds + Lt 1t
0
• Doob-Meyer decomposition for the continuous supermartingale S, and martingale representation theorem → there exists (Z, K) s.t. dSt = Zt dWt − dKt → Denote by Yt = St −
Rt 0
f (s)ds. Then (Y, Z) satisfies:
dYt = −f (t)dt + Zt dWt − dKt ,
YT = ξ
Yt ≥ Lt . • Consider the optimal stopping time for the Snell envelope, i.e. τt = inf{s ≥ t : St = Ht } ∧ T = inf{s ≥ t : Yt = Lt } ∧ T, which means that the stopped process St∧τt is a martingale. This implies that Kτt = Kt , i.e. T
Z
(Yt − Lt )dKt = 0. 0
I (Y, Z, K) is solution to the reflected BSDE (13)-(14)-(15).
• General case: (ξ, f ) satisfying standard assumption (H1) with Lipschitz generator, and L is a continuous obstacle in S2 (0, T ). I Existence and approximation by penalization For each n ∈ N, we consider the (unconstrained) BSDE Z T Z Ytn = ξ + f (s, Ysn , Zsn )ds + KTn − Ktn − t
where Ktn = n
RT t
T
Zsn .dWs ,
(17)
t
(Yns − Ls )− ds → existence and uniqueness of (Y n , Z n ).
• State a priori uniform estimates on the sequence (Y n , Z n , K n ): there exists ome positive constant C s.t. Z h E sup |Ytn |2 + 0≤t≤T
T
|Ztn |2 dt
+
|KTn |2
i
≤ C,
∀n ∈ N.
0
• By comparison principle for BSDE, (Yn )n is an increasing sequence, and it converges to some Y ∈ S2 (0, T ), and the convergence also holds in H2 (0, T ), i.e. hR i T n 2 limn→∞ E 0 |Yt − Yt | dt = 0. Moreover, Yt ≥ Lt . • Prove that (Z n , K n )n is a Cauchy sequence in H2 (0, T )d × S2 (0, T ): use Itˆo’s formula to |Ytn −Ytm |2 , and inequality 2ab ≤ εa2 + 1ε b2 for suitable ε. Consequently, (Z n , K n )n converges to some (Z, K) in H2 (0, T )d × S2 (0, T ). Pass to the limit in (17) in order to obtain the existence of (Y, Z, K) solution to the reflected BSDE.
Remark: Alternative formulation of the Skorokhod condition. The definition of a solution to the reflected BSDE (13)-(14) with the Skorokhod RT condition (15) 0 (Yt −Lt )dKt = 0 can be formulated equivalently in terms of minimal solution: • We say that (Y, Z, K) is a minimal solution to (13)-(14) if for any other solution ˜ K) ˜ solution to (13)-(14), we have Yt ≤ Y˜t , 0 ≤ t ≤ T . (Y˜ , Z, I Any solution to the reflected BSDE (13)-(14)-(15) is a minimal solution to (13)(14), and the converse is also true.
• Connection with variational inequalities in the Markov case Consider the case where: ξ = g(XT ), f (t, ω, Yt , Zt ) = f (t, Xt , Yt , Zt ), Lt = h(Xt ), 0 ≤ t ≤ T, with g ≥ h, and where X is a diffusion process on Rn dXt = b(Xt )dt + σ(Xt )dWt . I Then, the solution to the reflected BSDE is given by Yt = v(t, Xt ) for some deterministic function v, viscosity solution to the variational inequality: ∂v min − − Lv − f (t, x, v, σ 0 Dx v) , v − h = 0, ∂t v(T, .) = g
on [0, T ) × Rn on Rn .
VI. BSDE with constrained jumps and quasi-variational inequalities Consider the impulse control problem: T
Z h α v(t, x) = sup E g(XT ) + α
f (Xsα )ds t
X
+
i
c(Xτα− , ξi ) i
t<τi ≤T
with • controls: α = (τi , ξi )i where • (τi )i time decisions: nondecreasing sequence of stopping times • (ξi )i action decisions: sequence of r.v. s.t. ξi ∈ Fτi valued in E, • controlled process X α defined by Z s Z s X α α α γ(Xτα− , ξi ) Xs = x + b(Xu )du + σ(Xu )dWu + t
i
t
t<τi ≤s
I The corresponding dynamic programming equation is the quasi-variational inequality (QVI): ∂v − Lv − f , v − Hv = 0 , v(T, .) = g, min − ∂t where L is the second-order local operator: 1 Lv(t, x) = b(x).Dx v(t, x) + tr(σσ 0 (x)Dx2 v(t, x)) 2 and H is the nonlocal operator Hv(t, x) = sup He v(t, x) e∈E
with He v(t, x) = v(t, x + γ(x, e)) + c(x, e).
(18)
• The QVI (18) divides the time-space domain into: a continuation region C in which v(t, x) > Hv(t, x) and −
∂v − Lv − f = 0 ∂t
an action region D in which: v(t, x) = Hv(t, x) = sup v(t, x + γ(x, e)) + c(x, e). e∈E
• Various applications of impulse control problems: • Financial modelling with discrete transaction dates, due e.g. to fixed transaction costs or liquidity constraints • Optimal multiple stopping: swing options • Project’s investment and real options: management of power plants, valuation of gas storage and natural resources, forest management, ... • ... • Impulse control: widespread economical and financial setting with many practical applications → More generally to models with control policies that do not accumulate in time.
• Main theoretical and numerical difficulty in the QVI (18) : • The obstacle term contains the solution itself • It is nonlocal
I Classical approach : Decouple the QVI (18) by defining by iteration the sequence of functions (vn )n : i ∂vn+1 min − − Lvn+1 − f , vn+1 − Hvn = 0 , vn+1 (T, .) = g ∂t h
→ associated to a sequence of optimal stopping time problems (reflected BSDEs) → Furthermore, to compute vn+1 , we need to know vn on the whole domain → heavy computations, especially in high dimension (state space discretization): numerically challenging!
Idea of our approach • Instead of viewing the obstacle term as a reflection of v onto Hv (or vn+1 into Hvn ), I consider it as a constraint on the jumps of v(t, Xt ) for some suitable forward jump process X: • Let us introduce the uncontrolled jump diffusion X : Z dXt = b(Xt )dt + σ(Xt )dWt +
γ(Xt− , e)µ(dt, de), E
where µ is a Poisson random measure whose intensity λ is finite and supports the whole space E. → We randomize the state space! • Take some smooth function v(t, x) and define: Yt := v(t, Xt ),
Zt := σ(Xt− )0 Dx v(t, Xt− ),
Ut (e) := v(t, Xt− + γ(Xt− , e)) − v(t, Xt− ) + c(Xt− , e) = (He v − v)(t, Xt− )
I Apply Itˆo’s formula to Yt = v(t, Xt ) between t and T : Z T Z T Yt = YT + f (Xs )ds + KT − Kt − Zs .dWs t t Z TZ + [Us (e) − c(Xs− , e)]µ(ds, de), t
E
where Z Kt :=
t
(− 0
∂v − Lv − f )(s, Xs )ds ∂t
∂v − Lv − f, v − Hv] ≥ 0, and v(T, .) = g : ∂t I Then (Y, Z, U, K) satisfies Z T Z T Yt = g(XT ) + f (Xs )ds + KT − Kt − Zs .dWs t t Z TZ + [Us (e) − c(Xs− , e)]µ(ds, de), • Now, suppose that min[−
t
(19)
E
K is a nondecreasing process, and U satisfies the nonpositivity constraint : −Ut (e) ≥ 0, 0 ≤ t ≤ T, e ∈ E.
(20)
I View (19)-(20) as a Backward Stochastic Equation with jump constraints I We expect to retrieve the solution to the QVI (18) by solving the minimal solution to this constrained BSE.
General definition of BSDEs with constrained jumps Minimal Solution : find a solution (Y, Z, U, K) ∈ S2 × H2 (0, T )d × L2 (˜ µ) × A2 to Z T Z T Zs .dWs Yt = g(XT ) + f (Xs , Ys , Zs )ds + KT − Kt − t Z TZ t − (Us (e) − c(Xs− , Ys− , Zs , e))µ(ds, de) (21) t
E
with h(Ut (e), e) ≥ 0,
dP ⊗ dt ⊗ λ(de) a.e.
(22)
˜ U˜ , K) ˜ to (21)-(22) : such that for any other solution (Y˜ , Z, Yt ≤ Y˜t , 0 ≤ t ≤ T, a.s. Assumptions on the coefficients: • Forward SDE : b and σ Lipschitz continuous, γ bounded and Lipschitz continuous w.r.t. x uniformly in e: |γ(x, e) − γ(x0 , e)| ≤ k|x − x0 | ∀e ∈ E • Backward SDE : f , g and c have linear growth, f and g Lipschitz continuous, c Lipschitz continuous w.r.t. y and z uniformly in x and e |c(x, y, z, e) − c(x, y 0 , z 0 , e)| ≤ kc (|y − y 0 | + |z − z 0 |) • Constraint : h Lipschitz continuous w.r.t. u uniformly in e: |h(u, e) − h(u0 , e)| ≤ kh |u − u0 | and u 7→ h(u, e) nonincreasing. (e.g. h(u, e) = −u)
Existence and approximation via penalization Consider for each n the BSDE with jump: Z T Z T Ytn = g(XT ) + f (Xs , Ysn , Zsn )ds + KTn − Ktn − Zsn .dWs t Z TZ t − [Usn (e) − c(Xs− , Ysn− , Zsn , e)]µ(ds, de) t
(23)
E
with a penalization term Ktn = n
Z tZ 0
h− (Usn (e), e)λ(de)ds
E
where h− = max(−h, 0). → For each n, existence and uniqueness of (Y n , Z n , U n ) solution to (23) from Tang and Li (94), and Barles et al. (97). Convergence of the penalized solutions Theorem .1 Under (H1), there exists a unique minimal solution (Y, Z, U, K) ∈ S2 × H2 (0, T )d × L2 (˜ µ ) × A2 with K predictable, to (21)-(22). Y is the increasing limit of (Y n ) and also in S2 (0, T ), K is the weak limit of (K n ) in S2 (0, T ), and for any p ∈ [1, 2), kZ n − ZkHp (0,T ) + kU n − U kLp (˜µ) −→ 0, as n goes to infinity.
Sketch of proof. • Convergence of (Y n ): by comparison results (under the nondecreasing property of h) → Y n ≤ Y n+1 • Convergence of (Z n , U n , K n ) : more delicate! • A priori uniform estimates on (Y n , Z n , U n , K n )n in L2 → weak convergence of (Z n , U n , K n ) in L2 • Moreover, in general, we need some strong convergence to pass to the limit in the nonlinear terms f (X, Y n , Z n ), c(X, Y n , Z n ) and h(U n (e), e). → Control jumps of the predictable process K via a random partition of the interval (0,T) and obtain a convergence in measure of (Z n , U n , K n ) → Convergence of (Z n , U n , K n ) in Lp , p ∈ [1, 2)
Related semilinear QVI • By Markov property, the minimal solution to the constrained BSDE with jumps is Yt = v(t, Xt ) for some deterministic function v. • The function v is the unique viscosity solution to the QVI: h ∂v i 0 e min − − Lv − f (., v, σ Dx v), inf h(H v − v, e) = 0 e∈E ∂t together with the relaxed terminal condition: i e min v − g , inf h(H v − v, e) = 0 e∈E
on [0, T ) × Rn , (24)
on {T } × Rn .
(25)
I Probabilistic representation of semilinear QVIs, and in particular of impulse control problems by means of BSDEs with constrained jumps. I Numerical implications for the resolution of QVIs by means of simulation of the penalized BSDE: PhD thesis of M. Bernhart, in partnership with EDF for the valuation of swing options and gas storage contacts.
