OPTIMAL CONTROL APPLICATIONS AND METHODS Optim. Control Appl. Meth. 2005; 26:229–254 Published online 23 May 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/oca.759

Numerical solution to the optimal birth feedback control of a population dynamics: viscosity solution approach Bao-Zhu Guo1,2,n,y and Bing Sun3,4 1

Institute of Systems Science, Academy of Mathematics and System Sciences, Academia Sinica, Beijing 100080, People’s Republic of China 2 School of Computational and Applied Mathematics, University of the Witwatersrand, Private 3, Wits 2050, Johannesburg, South Africa 3 Institute of Systems Science, Academy of Mathematics and System Sciences, Academia Sinica, Beijing 100080, People’s Republic of China 4 Graduate School of the Chinese Academy of Sciences, Beijing 100049, People’s Republic of China

SUMMARY This paper is concerned with the optimal birth control of a McKendrick-type age-structured population dynamic system. We use the dynamic programming approach in our investigation. The Hamilton–Jacobi– Bellman equation satisfied by the value function is derived. It is shown that the value function is the viscosity solution of the Hamilton–Jacobi–Bellman equation. The optimal birth feedback control is found explicitly through the value function. A finite difference scheme is designed to obtain the numerical solution of the optimal birth feedback control. The validity of the optimality of the obtained control is verified numerically by comparing with different controls under the same constraint. All the data utilized in the computation are taken from the census of the population of China in 1989. Copyright # 2005 John Wiley & Sons, Ltd. KEY WORDS:

population dynamics; Hamilton–Jacobi–Bellman equation; viscosity solution; finite difference scheme; optimal feedback control

1. INTRODUCTION Optimal control problem is one of the central problems in modern control theory and seeking optimal controllers, particularly control laws in feedback form is argued as the Holy Grail of the optimal control theory [1]. It is well-known that the seminal contributions in optimal control theory are the Pontryagin maximum principle, the Bellman dynamic programming method, and the Kalman optimal linear regulator theory, all developed in the 1960s. Numerous works have n

Correspondence to: Bao-Zhu Guo, Institute of Systems Science, Academy of Mathematics and System Sciences, Academia Sinica, Beijing 100080, People’s Republic of China. y E-mail: [email protected] Contract/grant sponsor: National Natural Science Foundation of China Contract/grant sponsor: National Research Foundation of South Africa

Copyright # 2005 John Wiley & Sons, Ltd.

Received 16 August 2004 Revised 1 April 2005 Accepted 4 April 2005

230

B.-Z. GUO AND B. SUN

been done for the Pontryagin maximum principle for both finite-dimensional systems [2] and infinite-dimensional systems [3]. Unfortunately, the Pontryagin maximum principle provides only necessary conditions for the optimal control and it is usually not in feedback form. More seriously, ‘those sophisticated necessary conditions rarely give an insight into the structure of the optimal controls’ [4]. Nevertheless, due to the existence and uniqueness of the optimal control for most of practical problems, the Pontryagin maximum principle provides a possibility of seeking numerical solution for the optimal control at least in open-loop form. Basically, there are two ways of numerical methods solving optimal control problems through necessary conditions. It is generally believed that the indirect method that is mainly the multiple shooting method is the most powerful numerical method in seeking the optimal control through solving a two-point boundary-value problem obtained by the Pontryagin maximum principle. However, except for the complexity when the original problem involves inequality constraints of both state variables and controls, the big difficulty for shooting method is the ‘guess’ for the initial data to start the iterative numerical process. It demands that the user understands the essential of the problem well in physics, which is often not a trivial task [5]. For the direct method [6], the simplification for the original problem leads to the fall of the reliability and accuracy, and when the degree of discretization and parameterization is very high, the work of computation stands out and the solving process gives rise to ‘curse of dimensionality’ [5]. Nevertheless, over the last two decades, much progress has been made on numerical methods for solving optimal control problems. Teo et al. [7] dealt with computational methods for several general classes of optimal control problems subject to constraints. A finite difference scheme was developed for solving a class of hyperbolic partial differential equations in Reference [8]. Teo et al. [9] presented a powerful computational method, where the switching time points can also be considered as decision variables. In addition, many softwares on the optimal control are available now, such as MISER 3.2 [10], which was used to compute optimal controls in Reference [11] where the performance of the computed optimal feedback control is compared with the open-loop optimal control with the same initial condition. For other developments in computing the optimal control, we refer to References [12–16]. In contrast to the Pontryagin maximum principle that deals with one extremal at a time, the dynamic programming method deals with, on the other hand, families of extremals. Once the Hamilton–Jacobi–Bellman (HJB for short) equation satisfied by the value function is established, the feedback control law can be found by solving this first order non-linear partial differential equation. However, the new problem is raised immediately even from Pontryagin’s time that the HJB equation may have no classical solution no matter how smooth its coefficients are. The fundamental turn comes when the viscosity solution of the HJB equation was introduced in the 1980’s [17]. The advantage of viscosity solution is that this kind of weak solution not only exists but also is unique for most of HJB equations. Once this mathematical basis for the dynamic programming approach was rigorously established, the seeking of optimal (feedback) control becomes possible [18]. For infinite-dimensional optimal control problem by viscosity solution approach, we refer to References [19–23] and the references therein. Certainly, there are still other problems for seeking optimal control by solving viscosity solution of the HJB equation satisfied by the value function of the optimal control problem. One of the apparent problems is that in order to get the optimal feedback, we need not only the value function but also its gradient. The latter may not exist because the viscosity solution may not be differentiable. But for some big inertial systems, proper difference replacing the gradient of the value function still work for the numerical solution of the optimal feedback control. It Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

231

VISCOSITY SOLUTION APPROACH

was shown in Reference [24] that some simple difference scheme may produce numerically the viscosity solution. A recent interesting effort in solving numerically the HJB equation can be found in References [25–27] for some finite-dimensional control problems. In this paper, we study the optimal control problem for a McKendrick-type age-structured population dynamics by viscosity solution approach. This is an infinite-dimensional bilinear control problem. Many works have been done in the setting of partial differential equation models describing the population dynamics. In References [28,29], the optimal birth control problems of an age-structured population system of McKendrick-type were investigated and maximum principles for problems were established. The optimal control problem for a Gurtin– MacCamy-type system describing the evolution of an age-structured population was studied in Reference [30] and necessary optimality conditions were given. Anit-a [31] investigated an optimal harvesting problem for a non-linear age-dependent population system, and for some approximating problems, optimal controllers in feedback form were given by the dynamic programming method. Anit-a et al. [32] considered the optimal harvesting problem for the linear Lokta–McKendrick model with periodic vital rates and a periodic forcing term that sustains oscillations. Necessary optimality conditions were presented and a numerical algorithm was developed to approximate the optimal control and optimal harvest. The model in this paper describes the population evolution of human beings with birth control, which was developed in Reference [33]. The HJB equation satisfied by the value function will be derived. It is shown that the value function is the viscosity solution of the HJB equation. The optimal birth feedback control is thus found through the value function and its gradient. A finite difference scheme is designed to obtain the numerical solution of the optimal birth feedback control. The validity of the optimality of the obtained control is verified numerically by comparing with different controls under the same constraint. All the data utilized in the computation are taken from the census of the population of China in 1989 [34]. The paper is organized as follows. In Section 2, the dynamic programming principle for the value function of the optimal control problem is established. Some other properties of the solution as well as the continuity of the value function are presented. Section 3 is devoted to show that the value function is just the viscosity solution of the corresponding HJB equation. In Section 4, the optimal feedback control is formulated by the value function under the smooth assumption. In the last section, Section 5, we formulate a finite difference scheme to the HJB equation of the population system with linear quadratic optimal control. The numerical solution of optimal feedback control is presented. The validity of the optimality of the obtained control is verified numerically. 2. PROBLEM FORMULATION AND DYNAMIC PROGRAMMING PRINCIPLE A McKendrick-type model of age-structured population dynamics developed in Reference [33] is a first order partial differential equation with non-local boundary condition described by @pðr; tÞ @pðr; tÞ þ ¼ mðrÞpðr; tÞ; @t @r pðr; 0Þ ¼ p0 ðrÞ; pð0; tÞ ¼ bðtÞ

Z

05r5am ;

04r4am

t>0 ð1Þ

a2

bðrÞpðr; tÞ dr;

t50

a1

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

232

B.-Z. GUO AND B. SUN

where pðr; tÞ denotes the age density distribution at time t and age r for a closed population. mðrÞ is the relative mortality of the population, which is a non-negative measurable function and satisfies Z r Z am mðrÞ dr51 for r5am and mðrÞ dr ¼ 1 ð2Þ 0

0

am is the highest age ever attained by individuals of the population. bðrÞ ¼R kðrÞhðrÞ: kðrÞ is the a ratio of females and hðrÞ is the fertility pattern of females satisfying a12 hðrÞ dr ¼ 1: It is reasonable to assume that bðrÞ is bounded and measurable in ½a1 ; a2 ; the fecundity period of females, 05a1 5a2 5am ; and bðrÞ ¼ 0 outside ½a1 ; a2 : p0 ðrÞ is the initial distribution. bðtÞ is the specific fertility rate of the females at time t; which is considered to be the birth control of the population in macro-level. Let H ¼ L2 ð0; am Þ be the state space with the usual inner product h ; i and the inner product induced norm jj  jj: For any t > s50; let U½s; t ¼ fbðtÞ 2 ½b0 ; b1   Rþ jt 2 ½s; t; bðtÞ is measurable on ½s; tg Given T > 0 and p0 2 H; the optimal control problem is to find an optimal control bn ðÞ 2 U½0; T such that Jðbn Þ ¼ JðbÞ ¼

inf

bðÞ2U½0;T

Z

T

Z

0

JðbÞ

am

Lðpðr; tÞ; bðtÞ; r; tÞ dr dt þ

0

Z

ð3Þ

am

f0 ðr; pðr; TÞÞ dr 0

where pðr; tÞ is the solution to Equation (1) corresponding to bðÞ; and L; f0 satisfy Z am  Z am         4C1 þ C2 jjpjj 8ðt; bÞ 2 ½0; T  ½b0 ; b1 ; ; f ðr; pðrÞÞ dr LðpðrÞ; b; r; tÞ dr 0     0 0 Z am  Z am         ½ f0 ðr; p1 ðrÞÞ  f0 ðr; p2 ðrÞÞ dr;  ½Lðp1 ðrÞ; b; r; tÞ  Lðp2 ðrÞ; b; r; tÞ dr  0

0

4C3 jjp1  p2 jj Z

p2H

am

8ðt; bÞ 2 ½0; T  ½b0 ; b1 ; p1 ; p2 2 H

LðpðrÞ; b; r; tÞ dr is continuous in ðt; p; bÞ 2 Rþ  H  ½b0 ; b1 

ð4Þ

0

for some constants Ci ; i ¼ 1; 2; 3: Define the time-dependent operators Ab ðtÞ as follows: Ab ðtÞfðrÞ ¼ f0 ðrÞ  mðrÞfðrÞ 8f 2 DðAb ðtÞÞ  DðAb ðtÞÞ ¼

fðrÞ j f; Ab ðtÞf 2 H; fð0Þ ¼ bðtÞ

Z



a2

ð5Þ

bðrÞfðrÞ dr a1

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

233

VISCOSITY SOLUTION APPROACH

Then Equation (1) can be written as a first order evolution equation in H: @pðr; tÞ ¼ Ab ðtÞpðr; tÞ @t

ð6Þ

pðr; 0Þ ¼ p0 ðrÞ From now on, we also use Ab to denote the operator Ab ðtÞ when bðtÞ  b independent of time t; and the semigroup generated by Ab will be denoted as Tb ðtÞ: It is seen that Ab ðtÞ ¼ AbðtÞ : Define a family of evolution operators Tðt; s; bÞ; 04s4t51 by Rr 8  mðrÞ dr > > ; r5t  s fðr  t þ sÞe rtþs > > > Z a2 < Rt Rr  mðrÞ dr  mðrÞ dr Tðt; s; bÞfðrÞ ¼ bðt  rÞ bðtÞfðt  t þ s þ rÞe ttþsþr dte 0 ; r5t  s > > a1 > > > : 8f 2 H; 04t  s4a1   ½ðtsÞ=a   Y 1 ts Tðt; s; bÞ ¼ T t; s þ Tðs þ na1 ; s þ ðn  1Þa1 ; bÞ; t  s > a1 ð7Þ a1 ; b a1 n¼1 where ½x denotes the maximal integer not exceeding x: Tðt; s; bÞ is uniquely determined by fbðtÞ j t 2 ½s; tg: The following Theorem 1 comes from Reference [35], which can be verified directly by a straightforward computation. Theorem 1 (i) For each bðÞ 2 U½s; t; Tðt; s; bÞ is a bounded linear operator on H with Tðs; s; bÞ ¼ I; Tðt; s; bÞ ¼ Tðt; t0 ; bÞTðt0 ; s; bÞ

804s4t0 4t51

where I stands for the identity operator on H: (ii) Tð; ; bÞf is (strongly) continuous for each f 2 H uniformly in bðÞ 2 U½s; t: (iii) Let Tb1 ðtÞ be the C0 -semigroup generated by Ab1 on H: Then [33] jjTðt; s; bÞjj4jjTb1 ðtÞjj4MeoðtsÞ

for some M > 1 and o 2 R

(iv) The family of evolution operators fTðt; s; bÞg defined by (7) has the iterative relation 8 Rr  mðrÞdr > rtþs > fðr  t þ sÞe ; < Z Rr Tðt; s; bÞfðrÞ ¼ a2  mðrÞ dr > > 0 bðt  rÞ bðtÞTðt  r; s; bÞfðtÞ dte ; :

r5t  s ð8Þ r5t  s

a1

(v) If bðtÞ 2 C 1 ðs; 1Þ; then Tðt; s; bÞ is an evolution system, i.e. d Tðt; s; bÞp0 ¼ Ab ðtÞTðt; s; bÞp0 8p0 2 DðAb ð0ÞÞ dt Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

234

B.-Z. GUO AND B. SUN

(vi) For any t > s50 and any bi ðÞ 2 L2 ðs; tÞ ði ¼ 1; 2Þ jjTðt; s; b1 Þp0  Tðt; s; b2 Þp0 jj4jjbjj jjp0 jj jjb1 ðÞ  b2 ðÞjjL2 ðs;tÞ

ð9Þ

Theorem 2 Let Anb be the adjoint operator of Ab in H: Then there exists an operator C on H that is bounded, linear, self-adjoint and positive definite such that for each b 2 ½b0 ; b1 ; Anb C is a bounded linear operator on H and pffiffiffiffiffiffi sup jjAnb Cjj41 þ b1 jjbjj am ð10Þ b2½b0 ;b1 

Furthermore, the set Dn defined by Dn ¼ DðAnb Þ ¼ ffðrÞ j fðrÞ; f0 ðrÞ  mðrÞfðrÞ 2 H; fðam Þ ¼ 0g

ð11Þ

is independent of b 2 ½b0 ; b1  and dense in H: Proof Let the operator D be defined as DfðrÞ ¼ f0 ðrÞ  mðrÞfðrÞ ð12Þ DðDÞ ¼ ffðrÞ j f; Df 2 H; fðam Þ ¼ 0g Then   Rr d  mðrÞ dr fðrÞe 0 dr dr 0 Z am Rr Rr  mðrÞ dr  mðrÞ dr ¼ ½f0 ðrÞ  mðrÞfðrÞe 0 dr ¼ hDf; e 0 i

fð0Þ ¼

Z

am

0

It is easy to check that Anb fðrÞ ¼ f0 ðrÞ  mðrÞfðrÞ þ bbðrÞfð0Þ ¼ DfðrÞ  bbðrÞhDf; e



Rr 0

mðrÞ dr

i

ð13Þ

So, DðAnb Þ ¼ DðDÞ: Furthermore, let C ¼ ðI þ Dn DÞð1=2Þ : Then it follows from Theorem 13.13 of Reference [36] that C is a linear bounded self-adjoint positive operator and jjDCjj41: By (13), pffiffiffiffiffiffi jjAnb Cjj41 þ b1 jjbjj am This is the desired result.

&

Theorem 3 Let Dn be defined as (11) and let qðÞ 2 Dn : Then hqðÞ; pð; tÞi is differentiable almost everywhere on ½0; T and d hqðÞ; pð; tÞi ¼ hAnb ðtÞqðÞ; pð; tÞi; dt Copyright # 2005 John Wiley & Sons, Ltd.

t 2 ½0; T a:e:

ð14Þ

Optim. Control Appl. Meth. 2005; 26:229–254

235

VISCOSITY SOLUTION APPROACH

where pðr; tÞ ¼ Tðt; 0; bÞp0 ðrÞ for any p0 2 H and bðÞ 2 U½0; T: Anb ðtÞ ¼ AnbðtÞ is the adjoint operator of Ab ðtÞ: Proof Let t 2 ð0; TÞ: For simplicity, we only consider the right derivative. Let d > 0; t þ d5a1 : Then from (8), it has 8 Rr  mðrÞ dr > > ; r5d < pðr  d; tÞe td Z a2 Rt Rr pðr; t þ dÞ ¼  mðrÞ dr  mðrÞ dr > > bðtÞpðt  r þ d; tÞe trþd dte 0 ; r5d : bðd þ t  rÞ a1

So, hqðÞ; pð; t þ dÞ  pð; tÞi ¼

Z

am

qðrÞ½pðr  d; tÞe d

þ

Z



Rr td



d

qðrÞ bðd þ t  rÞ

Z

0

¼

Z

pðr; tÞ½qðr þ dÞe 0

Z

 pðr; tÞ dr

a2

bðtÞpðt  r þ d; tÞe

þ

Rt trþd

mðrÞ dr

dte



Rr

mðrÞ dr

0

R rþd r

mðrÞ dr

 qðrÞ dr 

Z

d

pðr; tÞqðrÞ dr 

Z

  pðr; tÞ dr

qðd þ t  rÞe



R dþtr 0

mðrÞ dr

Z

a2

bðtÞpðt þ r  t; tÞe

bðrÞ

am

qðrÞpðr; tÞ dr am d

0



t



dþt

Z



a1

am d



mðrÞ dr



Rt tþrt

mðrÞ dr

 dt dr

a1

d

qðrÞpðr; tÞ dr 0

¼

Z

am d

pðr; tÞ½qðr þ dÞe



R rþd r

mðrÞ dr

 qðrÞ dr 

Z

tþd

am

qðrÞpðr; tÞ dr am d

0

þ

Z

 qðd þ t  rÞe



R dþtr 0

mðrÞ dr

t

Z

a2

bðrÞ

bðtÞpðt þ r  t; tÞe



Rt tþrt

mðrÞ dr

 dt dr

a1

and hence

pð; t þ dÞ  pð; tÞ lim qðÞ; d#0 d Z am Z ¼ pðr; tÞ½q0 ðrÞ  mðrÞqðrÞ dr þ qð0ÞbðtÞ 0

a2

bðtÞpðt; tÞ dt

a1

¼ hAnb ðtÞqðÞ; pð; tÞi for all such t that the above limit exists, which are dense in ½0; T: Copyright # 2005 John Wiley & Sons, Ltd.

&

Optim. Control Appl. Meth. 2005; 26:229–254

236

B.-Z. GUO AND B. SUN

By virtue of Theorem 3, we will consider pðr; tÞ ¼ Tðt; 0; bÞp0 ðrÞ as the (weak) solution of (1) in the sense of (14), which is obtained by integrating (1) along the characteristics. The value function Vðt; p0 Þ for the optimal control problem is defined by Z T Z a m  Z am Vðt; p0 Þ ¼ inf Lðpðr; tÞ; bðtÞ; r; tÞ dr dt þ f0 ðr; pðr; TÞÞ dr ð15Þ bðÞ2U½t;T

0

t

0

where pðr; tÞ ¼ Tðt; t; bÞp0 ðrÞ is the solution of (1) corresponding to bðÞ 2 U½t; T: Theorem 4(Dynamic programming principle) For any 05d5T  t; Z tþd Z am  Vðt; p0 Þ ¼ inf Lðpðr; tÞ; bðtÞ; r; tÞ dr dt þ Vðt þ d; pð; t þ dÞÞ bðÞ2U½t;tþd

Z

VðT; p0 Þ ¼

0

t

ð16Þ

am

f0 ðr; p0 ðrÞÞ dr 0

where pðr; tÞ ¼ Tðt; t; bÞp0 ðrÞ: Proof Let WðtÞ ¼

Z

tþd

Z

am

inf

bðÞ2U½t;tþd

 Lðpðr; tÞ; bðtÞ; r; tÞ dr dt þ Vðt þ d; pð; t þ dÞÞ

0

t

# 2 U½t; T such that It is obvious that Vðt; p0 Þ4WðtÞ: For any given e > 0; there exists a bðÞ Z

T

Z

am

# Lð#pðr; tÞ; bðtÞ; r; tÞ dr dt þ

0

t

Z

am

f0 ðr; p# ðr; TÞÞ dr4Vðt; p0 Þ þ e

0

# 0 ðrÞ: Since where p# ðr; tÞ ¼ Tðt; t; bÞp Z T Z Vðt þ d; p# ð; t þ dÞÞ4 tþd

am

# Lð#pðr; tÞ; bðtÞ; r; tÞ dr dt þ

0

Z

am

f0 ðr; p# ðr; TÞÞ dr

0

it follows that Z

Z

tþd

am

WðtÞ4 t

Z

T

Z

4 t

# Lð#pðr; tÞ; bðtÞ; r; tÞ dr dt þ Vðt þ d; p# ð; t þ dÞÞ

0 am

# Lð#pðr; tÞ; bðtÞ; r; tÞ dr dt þ

0

Z

am

f0 ðr; p# ðr; TÞÞ dr4Vðt; p0 Þ þ e

0

By the arbitrariness of e; we obtain that WðtÞ4Vðt; p0 Þ: Therefore WðtÞ ¼ Vðt; p0 Þ: Copyright # 2005 John Wiley & Sons, Ltd.

&

Optim. Control Appl. Meth. 2005; 26:229–254

237

VISCOSITY SOLUTION APPROACH

Theorem 5 With M and o as in Theorem 1 and constants Ci ; i ¼ 1; 2; 3 in (4), the following assertions are true: (i) Vðt; p0 Þ4ð1 þ TÞC1 þ MC2 ð1 þ o1 ÞeoT jjp0 jj for all t 2 ½0; T; p0 2 H: (ii) Vðt; p1 Þ  Vðt; p2 Þj4ð1 þ o1 ÞMC3 eoT jjp1  p2 jj for all t 2 ½0; T; p1 ; p2 2 H: (iii) For any fixed p0 ; Vðt; p0 Þ is continuous in t: Proof For any bðÞ 2 U½t; T; Z

jVðt; p0 Þj4

T

ðC1 þ C2 jjTðt; t; bÞp0 jjÞ dt þ C1 þ C2 jjTðT; t; bÞp0 jj

t

M C2 eoT jjp0 jj þ C1 þ MC2 eoT jjp0 jj o   1 MC2 eoT jjp0 jj ¼ ð1 þ TÞC1 þ 1 þ o

4 C1 T þ

# 2 U½t; T such that This is (i). For any e > 0; take bðÞ Z T Z am Z # Vðt; p2 Þ5 Lð#p2 ðr; tÞ; bðtÞ; r; tÞ dr dt þ 0

t

am

f0 ðr; p# 2 ðr; TÞÞ dr  e

0

# 2 ðrÞ: Let p# 1 ðr; tÞ ¼ Tðt; t; bÞp # 1 ðrÞ: Then where p# 2 ðr; tÞ ¼ Tðt; t; bÞp Z T Z am Z am # Vðt; p1 Þ  Vðt; p2 Þ4 Lð#p1 ðr; tÞ; bðtÞ; r; tÞ dr dt þ f0 ðr; p# 1 ðr; TÞÞ dr t



0

Z

T

Z

0 am

# Lð#p2 ðr; tÞ; bðtÞ; r; tÞ dr dt 

0

t

Z

am

f0 ðr; p# 2 ðr; TÞÞ dr þ e

0

Z am    # #  ½Lð#p1 ðr; tÞ; bðtÞ; r; tÞ  Lð#p2 ðr; tÞ; bðtÞ; r; tÞ dr dt 4  0 t Z am    ½ f0 ðr; p# 1 ðr; TÞÞ  f0 ðr; p# 2 ðr; TÞÞ dr þ e þ  Z

T

0

Z 4 C3

T

jj#p1 ð; tÞ  p# 2 ð; tÞjj dt þ C3 jj#p1 ð; TÞ  p# 2 ð; TÞjj þ e

t

4

M C3 eoT jjp1  p2 jj þ C3 MeoT jjp1  p2 jj þ e ¼ ð1 þ o1 ÞMC3 eoT jjp1  p2 jj þ e o

Assertion (ii) is thus proved. Notice that in the last line above, we used the fact from (iii) of Theorem 1 that # jjp1  p2 jj4MeoðttÞ jjp1  p2 jj jj#p2 ð; tÞ  p# 2 ð; tÞjj4jjTðt; t; bÞjj Copyright # 2005 John Wiley & Sons, Ltd.

8t 2 ½t; T

Optim. Control Appl. Meth. 2005; 26:229–254

238

B.-Z. GUO AND B. SUN

% 2 U½t2 ; T such that Finally, for given p0 2 H and any t1 > t2 2 ½0; T; there exists a bðÞ Z T Z am Z am % Lðp% 2 ðr; tÞ; bðtÞ; r; tÞ dr dt þ f0 ðr; p% 2 ðr; TÞÞ dr  e Vðt2 ; p0 Þ5 0

t2

0

% 0 ðrÞ: Let p% 1 ðr; tÞ ¼ Tðt; t1 ; bÞp % 0 ðrÞ: Then where p% 2 ðr; tÞ ¼ Tðt; t2 ; bÞp Z T Z am Z am % Vðt1 ; p0 Þ  Vðt2 ; p0 Þ4 Lðp% 1 ðr; tÞ; bðtÞ; r; tÞ dr dt þ f0 ðr; p% 1 ðr; TÞÞ dr 0

t1



Z

T

Z

t1

Z

Z

T

Z

Z

am

0

t1

þ

am 0

t2

þ

am

0

t2

¼

0

Z

am

0

% Lðp% 2 ðr; tÞ; bðtÞ; r; tÞ dr dt 

Z

am 0

f0 ðr; p% 2 ðr; TÞÞ dr þ e

% Lðp% 2 ðr; tÞ; bðtÞ; r; tÞ dr dt % % ½Lðp% 1 ðr; tÞ; bðtÞ; r; tÞ  Lðp% 2 ðr; tÞ; bðtÞ; r; tÞ dr dt

½ f0 ðr; p% 1 ðr; TÞÞ  f0 ðr; p% 2 ðr; TÞÞ dr þ e

4 ðC1 þ C2 Me

oT

jjp0 jjÞðt1  t2 Þ þ C3

Z

T

% 0  Tðt; t2 ; bÞp % 0 jj dt jjTðt; t1 ; bÞp

t1

% 0  TðT; t2 ; bÞp % 0 jj þ e þ C3 jjTðT; t1 ; bÞp % 0  p0 jj þ e 4 ðC1 þ C2 MeoT jjp0 jjÞðt1  t2 Þ þ C3 ð1 þ TÞMeoT jjTðt1 ; t2 ; bÞp % 0  p0 jj ! 0: Now Hence, as t1  t2 ! 0; Vðt1 ; p0 Þ  Vðt2 ; p0 Þ ! 0; provided that jjTðt1 ; t2 ; bÞp assume t1  t2 5a1 : We have Rr Z am  mðrÞ dr % 0  p0 jj2 ¼ ½p0 ðr  t1 þ t2 Þe rt1 þt2  p0 ðrÞ2 dr jjTðt1 ; t2 ; bÞp t1 t2

Z

þ

t1 t2



% 1  rÞ bðt

Z

0

e



Z

a2

bðtÞp0 ðt  t1 þ t2 þ rÞ

a1

Rt tt1 þt2 þr

am

mðrÞ dr

dte



Rr 0

mðrÞ dr

2  p0 ðrÞ dr

½p0 ðr  t1 þ t2 Þ  p0 ðrÞ2 dr

42 t1 t2

þ2

Z

am

 R r  2  mðrÞ dr e rt1 þt2  1 p0 ðr  t1 þ t2 Þ dr

t1 t2

þ

2b21 jjbjjða2

þ2

Z

t1 t2

 a1 Þ1=2 jjp0 jj jt1  t2 j

jp0 ðrÞj2 dr ! 0

uniformly for b% as t1  t2 ! 0

0

So, (iii) is valid. The proof is complete. Copyright # 2005 John Wiley & Sons, Ltd.

& Optim. Control Appl. Meth. 2005; 26:229–254

VISCOSITY SOLUTION APPROACH

239

Remark 1 The proof of (iii) of Theorem 5 shows that for any fixed p0 ; jjTðt1 ; t2 ; bÞp0  p0 jj tends to zero uniformly for bðÞ 2 U½t2 ; t1  as t1  t2 ! 0: We will use this fact frequently in the sequel.

3. HJB EQUATION AND VISCOSITY SOLUTION For brevity in notation, we rewrite (6) as dPðtÞ ¼ Ab ðtÞPðtÞ dt

ð17Þ

Pð0Þ ¼ P where PðtÞ ¼ pð; tÞ; P ¼ p0 ðÞ: The value function is rewritten as Z T Vðt; PÞ ¼ inf f ðt; PðtÞ; bðtÞÞ dt þ cðPðTÞÞ bðÞ2U½t;T

where

t

PðtÞ ¼ Tðt; t; bÞP f ðt; PðtÞ; bðtÞÞ ¼ cðPðTÞÞ ¼

Z

am

Lðpðr; tÞ; bðtÞ; r; tÞ dr 0

Z

am

f0 ðr; pðr; TÞÞ dr 0

Theorem 6 If Vðt; PÞ 2 C 1 ð½0; T  HÞ; then V satisfies the following HJB equation: Vt ðt; PÞ þ inf fhVp ðt; PÞ; Ab Pi þ f ðt; P; bÞg ¼ 0 b2½b0 ;b1 

VðT; PÞ ¼ cðPÞ 8t 2 ½0; T; P 2

\

DðAb Þ

ð18Þ

b2½b0 ;b1 

Proof For any 05d5T  t and b 2 ½b0 ; b1 ; the dynamic programming principle implies that Z tþd Vðt; PÞ4 f ðt; Tb ðt  tÞP; bÞ dt þ Vðt þ d; Tb ðt þ dÞPÞ

ð19Þ

t

If P 2 DðAb Þ; since Tb ðtÞP ¼ P; it follows that P  Tb ðt þ dÞP ¼ P  ½P þ dAb P þ 8ðdÞ ¼ dAb P þ 8ðdÞ Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

240

B.-Z. GUO AND B. SUN

Hence, Vðt; PÞ  Vðt þ d; Tb ðt þ dÞPÞ ¼ Vt ðt; PÞd  hVp ðt; PÞ; Ab Pid þ 8ðdÞ from which (19) can be rewritten as 1 Vt ðt; PÞ  hVp ðt; PÞ; Ab Pi4 d

Z

tþd

f ðt; Tb ðt  tÞP; bÞ dt þ 8

t

ðdÞ d

Letting d ! 0; we obtain that Vt ðt; PÞ  hVp ðt; PÞ; Ab Pi4f ðt; P; bÞ So, Vt ðt; PÞ þ inf fhVp ðt; PÞ; Ab Pi þ f ðt; P; bÞg50 b2½b0 ;b1 

# 2 Uðt; t þ dÞ such that On the other hand, for any given e > 0; there exists a bðÞ Z tþd # # # þ dÞÞ  e f ðt; PðtÞ; bðtÞÞ dt þ Vðt þ d; Pðt Vðt; PÞ5

ð20Þ

t

# # where PðtÞ ¼ Tðt; t; bÞP: By (vi) of Theorem 1, we may assume without loss of generality that # 2 C 1 ½t; t þ d \ U½t; t þ d and bðtÞ # ¼ b for any given b 2 ½b ; b : If P 2 T bðÞ 1 2 b2½b0 ;b1  DðAb Þ; then # þ dÞ is differentiable with respect to d and Pðt d # Pðt þ dÞjd¼0 ¼ Ab# ðtÞP dd This together with (20) gives # Vt ðt; PÞ  hVp ðt; PÞ; Ab# ðtÞPi5f ðt; P; bðtÞÞ e i.e. # Vt ðt; PÞ þ hVp ðt; PÞ; AbðtÞ # Pi þ f ðt; P; bðtÞÞ4e So, Vt ðt; PÞ þ inf fhVp ðt; PÞ; Ab Pi þ f ðt; P; bÞg4e b2½b0 ;b1 

and hence Vt ðt; PÞ þ inf fhVp ðt; PÞ; Ab Pi þ f ðt; P; bÞg40 b2½b0 ;b1 

proving the theorem.

&

Remark 2 We shall denote D¼

\



0

DðAb Þ* fðrÞ; f ðrÞ þ mðrÞfðrÞ 2 H j fð0Þ ¼

b2½b0 ;b1 

Copyright # 2005 John Wiley & Sons, Ltd.

Z



a2

bðrÞfðrÞ dr ¼ 0

ð21Þ

a1

Optim. Control Appl. Meth. 2005; 26:229–254

VISCOSITY SOLUTION APPROACH

241

From now on, we always assume that Ab ðtÞ is dissipative for all b 2 ½b0 ; b1 : The assumption is reasonable since instead of (17), we may consider dQðtÞ ¼ ½Ab ðtÞ  CQðtÞ dt

ð22Þ

Qð0Þ ¼ P and then Ab ðtÞ  C with C ¼ 12 jjbjj2 is dissipative for each b 2 ½b0 ; b1  because RehAb ðtÞP; Pi412 jjbjj2 hP; Pi Since QðtÞ ¼ PðtÞeCt ; the value function becomes Z T  Vðt; PÞ ¼ inf f ðt; QðtÞeCt ; bðtÞÞ dt þ cðeCT QðTÞÞ bðÞ2U½t;T

t

where QðtÞ ¼ eCt Tðt; t; bÞP: First, we give a definition for the solution of the HJB equation (18) in the ‘viscosity sense’ [37]. Let O be an open set of H and set USCð½0; T   OÞ ¼ fupper-semicontinuous mappings u : ½0; T   O ! Rg LSCð½0; T   OÞ ¼ flower-semicontinuous mappings u : ½0; T   O ! Rg Definition 1 Uðt; PÞ 2 USCð½0; T  OÞ (respectively, Uðt; PÞ 2 LSCð½0; T   OÞÞ is a subsolution (respectively, supersolution) of (18) on ½0; T  O if for every test function F ¼ j þ g; j; jp 2 Cð½0; T  O; RÞ; g 2 C1 ðO; RÞ satisfying the condition (23) and the condition (24) below: 8 Rangeðjp Þ  Dn > > < the mapping ðt; PÞ ! hAnb jP ðt; PÞ; Pi > > : ð23Þ from ½0; T  O to R is equicontinuous in b 2 ½b0 ; b1  8 > there exists a h* : ½0; 1Þ ! R such that > > < h* is non-decreasing; h* 0 ð0Þ ¼ 0 and > > > : gðPÞ ¼ hðjjPjjÞ * 8P 2 H

ð24Þ

and for the local maximum point (respectively, the minimum point) ðt; PÞ of U  F (respectively, U þ F), we have jt ðt; PÞ þ inf fhAnb jp ðt; PÞ; Pi þ f ðt; P; bÞg50 b2½b0 ;b1 

Copyright # 2005 John Wiley & Sons, Ltd.

ð25Þ

Optim. Control Appl. Meth. 2005; 26:229–254

242

B.-Z. GUO AND B. SUN

(respectively, jt ðt; PÞ þ inf fhAnb jp ðt; PÞ; Pi þ f ðt; P; bÞg40Þ b2½b0 ;b1 

ð26Þ

Uðt; PÞ 2 Cð½0; T  OÞ is a viscosity solution of (18) if it is both a subsolution and a supersolution on ½0; T  O: Theorem 7 The value function Vðt; PÞ is a viscosity solution of the HJB equation (18). Proof Assume that ðt; PÞ is a local maximum point of V  F for the test function F: Then there exists a d > 0 such that Vðt; QÞ  Fðt; QÞ4Vðt; PÞ  Fðt; PÞ 8jt  tj; jjQ  Pjj5d i.e. Fðt; QÞ  Fðt; PÞ5Vðt; QÞ  Vðt; PÞ 8jt  tj; jjQ  Pjj5d By Remark 1, there exists a e > 0 such that for all b 2 ½b0 ; b1 ; Fðt þ d0 ; Tb ðd0 ÞPÞ  Fðt; PÞ5 Vðt þ d0 ; Tb ðd0 ÞPÞ  Vðt; PÞ 5

Z

tþd0

f ðt; Tb ðt  tÞP; bÞ dt

as 05d0 5e

t

The last inequality comes from Theorem 4. Notice that under conditions (23) and (24), we have

jðt þ d0 ; Tb ðd0 ÞQÞ  jðt; QÞ ¼

Z

d0 0

¼

Z

d0

jt ðt þ r; Tb ðrÞQÞ dr þ

0

Z 0

d jðt þ r; Tb ðrÞQÞ dr dr

d0

hAnb jp ðt þ r; Tb ðrÞQÞ; Tb ðrÞQi dr

ð27Þ

that holds for all t; d0 and all Q by first considering those Q 2 DðAb Þ and then for Q 2 H via the density argument. Also (24) implies that gðTb ðd0 ÞQÞ  gðQÞ40 Copyright # 2005 John Wiley & Sons, Ltd.

ð28Þ

Optim. Control Appl. Meth. 2005; 26:229–254

243

VISCOSITY SOLUTION APPROACH

for all Q 2 H; and all b 2 ½b0 ; b1  by first for Q 2 DðAb Þ and then via the density argument. Putting things all together, we have Z tþd0  f ðt; Tb ðt  tÞP; bÞ dt4 Fðt þ d0 ; Tb ðd0 ÞPÞ  Fðt; PÞ t

4 jðt þ d0 ; Tb ðd0 ÞPÞ  jðt; PÞ ¼

Z

d0

jt ðt þ t; Tb ðtÞPÞ dt 0

þ

Z

d0

hAnb jp ðt þ t; Tb ðtÞPÞ; Tb ðtÞPi dt

0

for all 05d0 5e: Dividing d0 on both sides above and letting d0 go to zero, we obtain jt ðt; PÞ þ hAnb jp ðt; PÞ; Pi þ f ðt; P; bÞ50 And hence (25) holds by the arbitrariness of b: Next, assume that ðt; PÞ is a local minimum point of V þ F for the test function F: Then there exists a e > 0 such that Fðt; PðtÞÞ þ Fðt; PÞ4Vðt; PðtÞÞ  Vðt; PÞ

ð29Þ

for all jt  tj5e and all admissible control bðÞ 2 U½t; t þ e; in which PðtÞ ¼ Tðt; t; bÞP: Let 05d0 5e: By Theorem 4,  Z tþd0  Vðt; PÞ ¼ inf f ðt; PðtÞ; bðtÞÞ dt þ Vðt þ d0 ; Pðt þ d0 ÞÞ bðÞ2U½t;tþd0 

t

Z 5

bðÞ2U½t;tþd0 



tþd0

inf

f ðt; PðtÞ; bðtÞÞ dt þ Vðt; PÞ  Fðt þ d0 ; Pðt þ d0 ÞÞ þ Fðt; PÞ

t

So, Z

f ðt; PðtÞ; bðtÞÞ dt  Fðt þ d0 ; Pðt þ d0 ÞÞ þ Fðt; PÞ 40

inf

bðÞ2U½t;tþd0 



tþd0

ð30Þ

t

holds for all 05d0 5e: As (27) and (28), we have Z d0 jðt þ d0 ; Pðt þ d0 ÞÞ  jðt; PÞ ¼ jt ðt þ t; Pðt þ tÞÞ dt 0

þ

Z

d0 0

hAnb ðtÞjp ðt þ t; Pðt þ tÞÞ; Pðt þ tÞi dt

and gðPðt þ d0 ÞÞ  gðPÞ40 for all bðÞ 2 U½t; t þ h: Then (30) implies that  Z tþd0  inf f ðt; PðtÞ; bðtÞÞ dt  jðt þ d0 ; Pðt þ d0 ÞÞ þ jðt; PÞ 40 bðÞ2U½t;tþd0 

t

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

244

B.-Z. GUO AND B. SUN

or Z

tþd0

f ðt; PðtÞ; bðtÞÞ dt 

inf

bðÞ2U½t;tþd0 

d0

jt ðt þ t; Pðt þ tÞÞ dt

0

t



Z

Z



d0 n

0

hAb ðtÞjp ðt þ t; Pðt þ tÞÞ; Pðt þ tÞi dt 40

ð31Þ

Suppose on the contrary that jt ðt; PÞ þ inf fhAnb jp ðt; PÞ; Pi þ f ðt; P; bÞg > 2W b2½b0 ;b1 

for some W > 0: Then for all b 2 ½b0 ; b1 ; jt ðt; PÞ  hAnb jp ðt; PÞ; Pi þ f ðt; P; bÞ > 2W By Remark 1, there exists a d 2 ð0; eÞ such that jt ðt; Pðt þ tÞÞ  hAnb ðtÞjp ðt; Pðt þ tÞÞ; Pðt þ tÞi þ f ðt þ t; Pðt þ tÞ; bðtÞÞ > W for all t5d and all bðÞ 2 U½t; t þ d: Contradicts to (31). So, V is a supersolution. The proof is complete. &

4. OPTIMAL FEEDBACK CONTROL A necessary condition as well as a characterization of optimal trajectory–control pairs can be formulated using the value function Vðt; PÞ: Theorem 8 Let Vðt; PÞ be the value function. Then for any trajectory–control pair ðPn ðtÞ; bn ðtÞÞ; Pn ðtÞ ¼ Tðt; 0; bn ÞP; the function Z T t ! Vðt; Pn ðtÞÞ  f ðt; Pn ðtÞ; bn ðtÞÞ dt ð32Þ t

is non-decreasing in ½0; T: Moreover, ðPn ðÞ; bn ðÞÞ is an optimal pair if and only if the above function is constant on ½0; T: Consequently, if Vðt; PÞ 2 C 1 ð½0; T  HÞ; bn ðÞ 2 C 1 ½0; T; Pn ð0Þ 2 DðAbn ð0Þ Þ; then ðPn ðÞ; bn ðÞÞ is an optimal pair if and only if Vt ðt; Pn ðtÞÞ þ hVp ðt; Pn ðtÞÞ; Abn ðtÞ Pn ðtÞi þ f ðt; Pn ðtÞ; bn ðtÞÞ ¼ 0 ð33Þ VðT; Pn ðTÞÞ ¼ cðPn ðTÞÞ

8t 2 ½0; T

Proof The first assertion is a direct consequence of the dynamic programming principle. The last assertion comes from the differentiation of (32) with respect to time t and (v) of Theorem 1. Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

VISCOSITY SOLUTION APPROACH

245

As for the second assertion, if ðPn ðtÞ; bn ðtÞÞ is not optimal then there exists another trajectory– control pair ðPðtÞ; bðtÞÞ; PðtÞ ¼ Tðt; 0; bÞP; such that Z

T

f ðt; PðtÞ; bðtÞÞ dt þ cðPðTÞÞ5

Z

0

T

f ðt; Pn ðtÞ; bn ðtÞÞ dt þ cðPn ðTÞÞ 0

and hence Vð0; PÞ ¼ Vð0; Pn ð0ÞÞ ¼ Vð0; Pð0ÞÞ4

Z Z

T

f ðt; PðtÞ; bðtÞÞ dt þ cðPðTÞÞ

0 T

f ðt; Pn ðtÞ; bn ðtÞÞ dt þ cðPn ðTÞÞ

5 0

¼

Z

T

f ðt; Pn ðtÞ; bn ðtÞÞ dt þ VðT; Pn ðTÞÞ

0

i.e. Vð0; Pn ð0ÞÞ 

Z

T

f ðt; Pn ðtÞ; bn ðtÞÞ dt5VðT; Pn ðTÞÞ

0

RT

n

Thus, Vðt; Pn ðtÞÞ  t f ðt; Pn ðtÞ; b ðtÞÞ dt is not constant on ½0; T: On the other hand, if it is not constant on ½0; T then the first assertion implies that Z

T

f ðt; Pn ðtÞ; bn ðtÞÞ dt þ cðPn ðTÞÞ

Vð0; PÞ5 0

RT Let e ¼ 0 f ðt; Pn ðtÞ; bn ðtÞÞ dt þ cðPn ðTÞÞ  Vð0; PÞ > 0: By the definition of the value function, there exists a ðPðtÞ; bðtÞÞ such that Z

T

f ðt; PðtÞ; bðtÞÞ dt þ cðPðTÞÞ  e

Vð0; PÞ > 0

Hence, Z

T

f ðt; PðtÞ; bðtÞÞ dt þ cðPðTÞÞ5 0

Z

T

f ðt; Pn ðtÞ; bn ðtÞÞ dt þ cðPn ðTÞÞ 0

which implies that ðPn ðtÞ; bn ðtÞÞ is not optimal, proving the required result.

&

The last assertion of Theorem 8 will lead to the classical verification theorem, which plays an important role in testing for optimality of a given admissible pair, and more importantly, in constructing optimal feedback controls [38,39]. In fact, suppose the value function Vðt; PÞ is smooth. Since Vðt; PÞ is a viscosity solution of the HJB equation, we have Vt ðt; PÞ þ inf fhVp ðt; PÞ; Ab Pi þ f ðt; P; bÞg ¼ 0 b2½b0 ;b1 

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

246

B.-Z. GUO AND B. SUN

then the condition of optimality (33) is equivalent to hVp ðt; Pn ðtÞÞ; Abn ðtÞ Pn ðtÞi þ f ðt; Pn ðtÞ; bn ðtÞÞ ¼ inf fhVp ðt; Pn ðtÞÞ; Ab Pn ðtÞi þ f ðt; Pn ðtÞ; bÞg b2½b0 ;b1 

:¼ HðPn ðtÞ; Vp ðt; Pn ðtÞÞÞ

for t 2 ½0; T a:e:

ð34Þ

Now, let SðZÞ :¼ arg min fhVz ðt; ZÞ; Ab Zi þ f ðt; Z; bÞg b2½b0 ;b1 

¼ fb 2 ½b0 ; b1  j HðZ; Vz ðt; ZÞÞ ¼ hVz ðt; ZÞ; Ab Zi þ f ðt; Z; bÞg

ð35Þ

Then (34) says that bn ðÞ 2 U½0; T is an optimal control for the initial state p0 if and only if bn ðtÞ 2 SðPn ðtÞÞ

for t 2 ½0; T a:e:

ð36Þ

where Pn ðtÞ ¼ Tðt; 0; bn Þp0 : The formula for finding the optimal feedback control is (36). It should be indicated that (36) is true only when the value function is differentiable. However, this is not true usually in the sense of analyticity. Nevertheless, it is still an important formula to construct numerically the optimal feedback control, which will be tested in the next section.

5. FINITE DIFFERENCE SCHEME FOR THE NUMERICAL SOLUTION OF OPTIMAL FEEDBACK CONTROL In this section, we will limit ourselves to the following linear quadratic optimal control problem: Jðbn Þ ¼

inf

bðÞ2U½0;T

JðbÞ

1 ¼ inf bðÞ2U½0;T 2

Z

T

% 2 dt þ ½bðtÞ  bðtÞ

0

Z

am

0

2

½pðr; TÞ  p% ðrÞ dr

 ð37Þ

% denotes the mean critical fertility rate of females in an ideal society and p% ðrÞ denotes where bðtÞ the stable age density distribution of the population with the zero increasing rate. Corresponding to (3) and (18), we have 1 % 2; Lðpðr; tÞ; bðtÞ; r; tÞ ¼ ½bðtÞ  bðtÞ 2 f ðt; pð; tÞ; bðtÞÞ ¼

am % 2; ½bðtÞ  bðtÞ 2

Copyright # 2005 John Wiley & Sons, Ltd.

1 f0 ðr; pðr; TÞÞ ¼ ½pðr; TÞ  p% ðrÞ2 2 Z am 1 ½pðr; TÞ  p% ðrÞ2 dr cðpð; TÞÞ ¼ 2 0 Optim. Control Appl. Meth. 2005; 26:229–254

247

VISCOSITY SOLUTION APPROACH

and the HJB equation can be rewritten as n o am % 2 ¼0 ½b  bðtÞ hVp ðt; pÞ; Ab pi þ Vt ðt; pÞ þ inf b2½b0 ;b1  2 Z am \ 1 ½pðrÞ  p% ðrÞ2 dr 8t 2 ½0; T; p 2 DðAb Þ VðT; pÞ ¼ 2 0 b2½b ;b  0

ð38Þ

1

Now we come to solve the above equation numerically. To do so, let tj ¼ T þ jDt; j ¼ 0; 1; 2; . . . ; N in which Dt ¼ T=N and N is an integer. For given e > 0; we use the following approximation: hVp ðt; pÞ; Ab pi ¼

    Ab p jjAb pjj Ab p jjAb pjj  V t; p þ e Vp ðt; pÞ; e  Vðt; pÞ jjAb pjj e jjAb pjj e

For the initial state p0 let pi ¼ pi1 þ ðAb pi1 =jjAb pi1 jjÞ e; i ¼ 1; 2; . . . ; M: Approximate (38) by the difference equation: j Vijþ1  Vij Vij  Vi1 am j % j 2 þ jjAb j pi jj þ ½b  b  ¼ 0 i Dt e 2 i ( ) jþ1 Vijþ1  Vi1 am jþ1 jþ1 2 % bi 2 arg inf þ jjAb pi jj þ ½b  b  b2½b0 ;b1  e 2

ð39Þ

for i ¼ 1; 2; . . . ; M and j ¼ 0; 1; 2; . . . ; N; where Vij  Vðtj ; pi Þ: Referring to Li and Feng [40], the following sufficient condition for the stability of the difference scheme will be assumed: jDtj max jjAb pi jj41 e 14i4M

ð40Þ

Set aij ¼ ðDt=eÞjjAb j pi jj: We give the algorithm for the numerical solution of (38) as follows. i

Algorithm of solving the HJB equation. Step 1: Initialization. Set Vi0 ¼ VðT; pi Þ ¼ cðpi Þ; b0i 2 arg inf



b2½b0 ;b1 

pi ¼ pi1 þ

cðpi Þ  cðpi1 Þ am jjAb pi jj þ ½b  b% 0 2 e 2

:¼ arg inf fHðb; pÞg; b2½b0 ;b1 

Copyright # 2005 John Wiley & Sons, Ltd.

Ab pi1 e jjAb pi1 jj

i ¼ 1; 2; . . . ; M



ð41Þ

Optim. Control Appl. Meth. 2005; 26:229–254

248

B.-Z. GUO AND B. SUN

Here the formula for b0i comes from

n

bðt; pðÞÞ 2 arg inf

b2½b0 ;b1 

hVp ðt; pÞ; Ab pi þ

o am % 2 ½b  bðtÞ 2

ð42Þ

Step 2: Iteration. By (39), j Vijþ1 ¼ ð1  aij ÞVij þ aji Vi1 

( bjþ1 i

2 arg inf

b2½b0 ;b1 

am j % j 2 ðb  b Þ Dt 2 i

jþ1 Vijþ1  Vi1 am jjAb pi jj þ ½b  b% jþ1 2 e 2

)

for i ¼ 1; 2; . . . ; M and j ¼ 0; 1; 2; . . . ; N  1: It is seen that 8 b1 if Hðb1 ; pÞ ¼ minfHðb1 ; pÞ; Hðb0 ; pÞ; Hðb% 0 ; pÞg > > < b0i ¼ b0 if Hðb0 ; pÞ ¼ minfHðb1 ; pÞ; Hðb0 ; pÞ; Hðb% 0 ; pÞg > > : *0 b if Hðb* 0 ; pÞ ¼ minfHðb1 ; pÞ; Hðb0 ; pÞ; Hðb% 0 ; pÞg

ð43Þ

ð44Þ

for i ¼ 1; 2; . . . ; M: The similar formula can be also obtained for bji ; i ¼ 1; 2; . . . ; M; j ¼ 1; 2; . . . ; N  1: From (42), the optimal feedback control is n o am % 2 ½b  bðtÞ bnp0 ðt; pn ð; tÞÞ 2 arg inf hVp ðt; pn ð; tÞÞ; Ab pn ð; tÞi þ ð45Þ b2½b0 ;b1  2 in which pn ð; tÞ is the optimal trajectory of the system. Because it involves the optimal trajectory in (45), we need to find the solution of (1). Here we utilize the difference scheme to compute the approximate solution of the initial value problem (1) (see Reference [33]). Adopting the equally spaced grid method, we have pi;j  pi1;j pi;j  pi;j1 þ ¼ mi pi;j ; Ds Ds pi;0 ¼ p0 ðiDsÞ; p0;j ¼ bj Ds

a2 X

14j4J; 14i42K

04i42K bi pi; j ;

ð46Þ

14j4J

i¼a1

in which bi ¼ bðiDsÞ; bj ¼ bð jDsÞ; i ¼ a1 ; . . . ; a2 ; 2KDs ¼ am ; JDs ¼ T: It follows that pi; j ¼

pi1; j þ pi; j1 ; 2 þ mi Ds

pi;0 ¼ p0 ðiDsÞ; p0;j ¼ bj

a2 X

14j4J; 14i42K

04i42K

bi pi;j Ds;

ð47Þ

14j4J

i¼a1

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

249

VISCOSITY SOLUTION APPROACH

Next we give steps of finding the optimal feedback control in detail. It focuses on the solving the difference scheme (47) to obtain the optimal trajectory. Moreover, in the process, the algorithm of solving the HJB equation will be called frequently to get the corresponding feedback control function. Steps of finding the optimal feedback control. Step 1: Call the algorithm of solving the HJB equation to get the feedback control function bðt; p0 Þ: Substitute bð0; p0 Þ into (47) to get the optimal trajectory pn ð; s1 Þ; s1 ¼ Ds: Step 2: Take pn ð; s1 Þ as the initial data p0 as in the first step, and call the algorithm of solving the HJB equation again to get the feedback control bðs1 ; pn ð; s1 ÞÞ: Substitute bðs1 ; pn ð; s1 ÞÞ into (47) to get the optimal trajectory pn ð; s2 Þ; s2 ¼ 2Ds: Step 3: Repeating the processes above, we get all feedback control functions bðsj ; pn ð; sj ÞÞ; sj ¼ jDs; j ¼ 0; 1; . . . ; J; that is to say, bnp0 ðt; pn ð; tÞÞ ¼ fbð0; p0 ðÞÞ; bðs1 ; pn ð; s1 ÞÞ; bðs2 ; pn ð; s2 ÞÞ; . . . ; bðT; pn ð; TÞÞg

ð48Þ

which is the optimal feedback control. Now we are in a position to find the numerical solution of linear quadratic optimal control for the Chinese population based on the scheme (41), (43) and (47). The initial data are extracted from the Chinese population sampling census in 1989 [34], which are plotted by MATLAB 6.1 as Figure 4 (age-structure of females); Figure 5 (the relative mortality); Figure 6 (the age-structure of the total population). The age-structure of an ideal society taken from Reference [33] are listed in Table I, which is used to get p% ðrÞ by multiplying the proportion in the table with the total ideal population Nsum ¼ 1 400 000: The fertility pattern hðrÞ is approximated by the Gamma density distribution curve in statistics [33] in which the peak value of the fertility age is assumed to be 24. Other parameters are listed as follows: T ¼ 25;

b0 ¼ 1;

a1 ¼ 15;

a2 ¼ 49; am ¼ 99;

b1 ¼ 3;

% ¼2 bðtÞ e ¼ 0:01

Table I. The age structure of an ideal society. Age, r

Proportion

Age, r

Proportion

0 1–5 6–10 11–15 16–20 21–25 26–30 31–35 36–40 41–45

0.013 0.065 0.065 0.065 0.065 0.065 0.065 0.065 0.064 0.064

46–50 51–55 56–60 61–65 66–70 71–75 76–80 81–85 > 85

0.063 0.062 0.061 0.057 0.053 0.047 0.037 0.024 0.003

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

250

B.-Z. GUO AND B. SUN

In addition, due to the limitation of computer memory, the data in Figure 4 and Figure 6 and Nsum are divided by 10 000. All computations are performed in Visual C++ 6.0 and numerical results are plotted by MATLAB 6.1. Figure 1 is the value function Vðt; pn ð; tÞÞ and Figure 2 is the optimal feedback control bnp0 ðt; pn ð; tÞÞ: To end the paper, we have to check the optimality of the numerical solution of optimal feedback control. This is done by comparing the cost functional Jðbnp0 ðt; pn ð; tÞÞÞ of obtained optimal control–trajectory pair with that of arbitrarily chosen control bp0 ðtÞ; Jðbp0 ðtÞÞ under the

value function : v(t,p*(.,t))

22

x 10 5

v

4 3 2 1 0 0

5

10

15

t

20

25

Figure 1. The value function Vðt; pn ð; tÞÞ:

optimal feedback control : β*p0(t,p*(.,t)) 3 2.8 2.6 2.4 β

2.2 2 1.8 1.6 1.4 1.2 1 0

5

10

15

20

25

t

Figure 2. The optimal feedback control bnp0 ðt; pn ð; tÞÞ: Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

VISCOSITY SOLUTION APPROACH

251

Jðbnp0 ðt; pn ð; tÞÞÞ4Jðbp0 ðtÞÞ

ð49Þ

same initial condition, that is,

Referring to Figure 3, we compute these costs Jðbi Þ ði ¼ 1; 2; . . . ; 7Þ for seven different controls bi 2 U½0; T; respectively. The cost value Jðbnp0 ðt; pn ð; tÞÞÞ is also computed. These results are listed in Table II. It is seen from Table II that for the optimal feedback control bnp0 ðt; pn ð; tÞÞ; Jðbnp0 ðt; pn ð; tÞÞÞ ¼ 11641607717:814667; which is evidently less than other costs Jðbi Þ; i ¼ 1; 2; . . . ; 7: In other words, we do get the numerical solution of optimal birth feedback control for the Chinese population from 1989 to 2014.

β1(t)

β2(t)

3

3

2

2

1

1 0

5

10

15

20

25

0

5

10

β3(t) 3

2

2

1

1 5

10

15

20

25

0

5

10

β5(t) 3

2

2

1

1 5

10

15

20

25

0

5

7

3

3

2

2

1

1 5

10

15

20

25

10

15

20

25

β*p0 (t,p* (.,t))

β (t)

0

25

β6(t)

3

0

20

β4(t)

3

0

15

15

20

25

0

5

10

15

20

25 n

Figure 3. Seven different arbitrarily chosen control bi and the optimal feedback control bp0 ðt; pn ð; tÞÞ: Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

252

B.-Z. GUO AND B. SUN

Table II. Different b and its corresponding cost JðbÞ: b b1 ðtÞ b2 ðtÞ b3 ðtÞ b4 ðtÞ

JðbÞ

b

JðbÞ

11641608005.975365 11641608005.975365 11641607901.862444 11641608172.639233

b5 ðtÞ b6 ðtÞ b7 ðtÞ bnp0 ðt; pn ð; tÞÞ

11641608172.639233 11641608172.639233 11641608172.639233 11641607717.814667

female number

15000

10000

5000

0 0

10

20

30

40

50 60 age

70

80

90 100

Figure 4. The age-structure of females of Chinese population in 1989.

relative mortality

0.5

µ(r)

0.4 0.3 0.2 0.1 0 0

10

20

30

40

50 r

60

70

80

90 100

Figure 5. The relative mortality of Chinese population in 1989. Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

253

VISCOSITY SOLUTION APPROACH

4

Initial density distribution

x 10 3 2.5

p0(r)

2 1.5 1 0.5 0 0

10

20

30

40

50 r

60

70

80

90

100

Figure 6. The age-structure of Chinese population in 1989.

ACKNOWLEDGEMENTS

The authors would like to thank an anonymous reviewer who indicates many recent developments in numerical solutions to optimal control problems, particularly, the software MISER of Reference [11] may be better to be used for performance comparisons. This work was supported by the National Natural Science Foundation of China and the National Research Foundation of South Africa.

REFERENCES 1. Sussmann HJ, Willems JC. 300 years of optimal control: from the brachystochrone to the maximum principle. IEEE Control Systems Magazine 1997; 17(3):32–44. 2. Girsanov IV. Lectures on Mathematical Theory of Extremum Problems. Lecture Notes in Economics and Mathematical Systems, vol. 67. Springer: Berlin, 1972. 3. Li X, Yong J. Optimal Control Theory for Infinite Dimensional Systems. Birkha¨user: Boston, 1995. 4. Rubio JE. Control and Optimization: The Linear Treatment of Nonlinear Problems. Nonlinear Science: Theory and Applications. Manchester University Press: Manchester, 1986. 5. Bryson Jr AE. Optimal control}1950 to 1985. IEEE Control Systems Magazine 1996; 13(3):26–33. 6. von Stryk O, Bulirsch R. Direct and indirect methods for trajectory optimization. Annals of Operations Research 1992; 37(1–4):357–373. 7. Teo KL, Goh CJ, Wong KH. A Unified Computational Approach to Optimal Control Problems. Longman Scientific and Technical: New York, 1991. 8. Rehbock V, Wang S, Teo KL. Computing optimal control with hyperbolic partial differential equation. Journal of the Australian Mathematical Society Series B 1998; 40(2):266–287. 9. Teo KL, Jennings LS, Lee HWJ, Rehbock V. The control parametrization enhancing transform for constrained optimal control problems. Journal of the Australian Mathematical Society Series B 1999; 40:314–335. 10. Jennings LS, Teo KL, Goh CJ. MISER3.2}Optimal Control Software: Theory and User Manual. Department of Mathematics, The University of Western Australia, Australia, 1997. 11. Rehbock V, Teo KL, Jennings LS. Suboptimal feedback control for a class of nonlinear systems using spline interpolation. Discrete and Continuous Dynamical Systems 1995; 1(2):223–236. 12. Augustin D, Maurer H. Computational sensitivity analysis for state constrained optimal control problems. Annals of Operations Research 2001; 101:75–99. 13. Liu Y, Ito S, Lee HWJ, Teo KL. Semi-infinite programming approach to continuously-constrained linear-quadratic optimal control problems. Journal of Optimization Theory and Applications 2001; 108(3):617–632. Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

254

B.-Z. GUO AND B. SUN

14. Malanowski K, Maurer H, Pickenhain S. Second-order sufficient conditions for state-constrained optimal control problems. Journal of Optimization Theory and Applications 2004; 123(3):595–617. 15. Maurer H, Oberle HJ. Second order sufficient conditions for optimal control problems with free final time: the Riccati approach. SIAM Journal on Control and Optimization 2002; 41(2):380–403. 16. Maurer H, Osmolovskii NP. Second order sufficient conditions for time-optimal bang–bang control. SIAM Journal on Control and Optimization 2004; 42(6):2239–2263. 17. Crandall MG. Viscosity solutions: a primer. In Viscosity Solutions and Applications, Dolcetta IC, Lions PL (eds). Lecture Notes in Mathematics, vol. 1660. Springer: Berlin, 1997; 1–43. 18. Bardi M. Some applications of viscosity solutions to optimal control and differential games. In Viscosity Solutions and Applications, Dolcetta IC, Lions PL (eds). Lecture Notes in Mathematics, vol. 1660. Springer: Berlin, 1997; 44–97. 19. Barron EN. Application of viscosity solutions of infinite-dimensional Hamilton–Jacobi–Bellman equations to some problems in distributed optimal control. Journal of Optimization Theory and Applications 1990; 64(2):245–268. 20. Cannarsa P, Gozzi F, Soner HM. A dynamic programming approach to nonlinear boundary control problems of parabolic type. Journal of Functional Analysis 1993; 117(1):25–61. 21. Gozzi F, Sritharan SS, S´wie-ch A. Viscosity solutions of dynamic-programming equations for the optimal control of the two-dimensional Navier–Stokes equations. Archives for Rational Mechanics and Analysis 2002; 163(4):295–327. 22. Kocan M, Soravia P. A viscosity approach to infinite-dimensional Hamilton–Jacobi equations arising in optimal control with state constraints. SIAM Journal on Control and Optimization 1998; 36(4):1348–1375. 23. Yong JM, Zhou XY. Stochastic Controls: Hamiltonian Systems and HJB Equations. Applications of Mathematics, vol. 43. Springer: New York, 1999. 24. Fleming WH, Soner HM. Controlled Markov Processes and Viscosity Solutions. Applications of Mathematics, vol. 25. Springer: New York, 1993. 25. Huang CS, Wang S, Teo KL. On application of an alternating direction method to Hamilton–Jacobi–Bellman equations. Journal of Computational and Applied Mathematics 2004; 166(1):153–166. 26. Wang S, Gao F, Teo KL. An upwind finite-difference method for the approximation of viscosity solutions to Hamilton–Jacobi–Bellman equations. IMA Journal of Mathematical Control and Information 2000; 17(2):167–178. 27. Wang S, Jennings LS, Teo KL. Numerical solution of Hamilton–Jacobi–Bellman equations by an upwind finite volume method. Journal of Global Optimization 2003; 27(2–3):177–192. 28. Chan WL, Guo BZ. Optimal birth control of population dynamics. Journal of Mathematical Analysis and Applications 1989; 144(2):532–552. 29. Chan WL, Guo BZ. Optimal birth control of population dynamics. II. Problems with free final time, phase constraints, and mini-max costs. Journal of Mathematical Analysis and Applications 1990; 146(2):523–539. 30. Barbu V, Iannelli M. Optimal control of population dynamics. Journal of Optimization Theory and Applications 1999; 102(1):1–14. 31. Anit-a S. Optimal harvesting for a nonlinear age-dependent population dynamics. Journal of Mathematical Analysis and Applications 1998; 226(1):6–22. 32. Anit-a S, Iannelli M, Kim M-Y, Park E-J. Optimal harvesting for periodic age-dependent population dynamics. SIAM Journal on Applied Mathematics 1998; 58(5):1648–1666. 33. Song J, Yu JY. Population System Control. Springer: Berlin, 1988. 34. Department of Demographic Census in National Bureau of Statistics of China. China Population Statistics Yearbook. Scientific and Technical Documents Publishing House: Beijing, 1990 (in Chinese). 35. Guo BZ, Yao CZ. New results on the exponential stability of non-stationary population dynamics. Acta Mathematica Scientia (English edn) 1996; 16(3):330–337. 36. Rudin W. Functional Analysis (2nd edn). McGraw-Hill: New York, 1991. 37. Yung SP. Multiprocess optimal control problem on Hilbert space and related Hamilton–Jacobi equation. Nonlinear Analysis 1997; 29(11):1319–1342. 38. Gozzi F, S´wie-ch A, Zhou XY. A corrected proof of the stochastic verification theorem within the framework of viscosity solutions. SIAM Journal on Control and Optimization 2005; 43(6):2009–2019. 39. Zhou XY. Verification theorems within the framework of viscosity solutions. Journal of Mathematical Analysis and Applications 1993; 177(1):208–225. 40. Li RH, Feng GC. Numerical Solution of Differential Equations. Higher Education Press: Shanghai, 1980 (in Chinese).

Copyright # 2005 John Wiley & Sons, Ltd.

Optim. Control Appl. Meth. 2005; 26:229–254

Numerical solution to the optimal birth feedback ... - Semantic Scholar

Published online 23 May 2005 in Wiley InterScience ... 4 Graduate School of the Chinese Academy of Sciences, Beijing 100049, People's ... the degree of discretization and parameterization is very high, the work of computation stands out and ...

241KB Sizes 1 Downloads 250 Views

Recommend Documents

Numerical solution to the optimal birth feedback ... - Semantic Scholar
May 23, 2005 - of a population dynamics: viscosity solution approach ...... Seven different arbitrarily chosen control bi and the optimal feedback control bn.

Numerical solution to the optimal feedback control of ... - Springer Link
Received: 6 April 2005 / Accepted: 6 December 2006 / Published online: 11 ... of the continuous casting process in the secondary cooling zone with water spray control ... Academy of Mathematics and System Sciences, Academia Sinica, Beijing 100080, ..

Development of Intuitive and Numerical ... - Semantic Scholar
Dec 27, 1990 - Our study explored the relationship between intuitive and numerical propor- .... Table 1. Component Patterns for the Fuzzy Set Prototypes.

Development of Intuitive and Numerical ... - Semantic Scholar
Dec 27, 1990 - were asked to predict the temperature of a container of water produced by combining two separate containers at different temperatures. The same problems were presented in two ways. In the numerical condition the use of quantitative or

OSNAP: Faster numerical linear algebra ... - Semantic Scholar
in a low rank matrix A have been revealed, and the goal is to then recover A. This ...... Disc. Math. and Theor. Comp. Sci., AC:145–154, 2003. [30] Bo'az Klartag ...

Optimal Detection of Heterogeneous and ... - Semantic Scholar
Oct 28, 2010 - where ¯Φ = 1 − Φ is the survival function of N(0,1). Second, sort the .... (β;σ) is a function of β and ...... When σ ≥ 1, the exponent is a convex.

Optimal Allocation Mechanisms with Single ... - Semantic Scholar
Oct 18, 2010 - [25] Milgrom, P. (1996): “Procuring Universal Service: Putting Auction Theory to Work,” Lecture at the Royal Academy of Sciences. [26] Myerson ...

Genetically Evolving Optimal Neural Networks - Semantic Scholar
Nov 20, 2005 - URL citeseer.ist.psu.edu/curran02applying.html. [4] B.-T. Zhang, H. Mühlenbein, Evolving optimal neural networks using genetic algorithms.

Optimal Dynamic Hedging of Cliquets - Semantic Scholar
May 1, 2008 - Kapoor, V., L. Cheung, C. Howley, Equity securitization; Risk & Value, Special Report, Structured. Finance, Standard & Poor's, (2003). Laurent, J.-P., H. Pham, Dynamic Programming and mean-variance hedging, Finance and Stochastics, 3, 8

Genetically Evolving Optimal Neural Networks - Semantic Scholar
Nov 20, 2005 - Genetic algorithms seem a natural fit for this type .... For approaches that address network architecture, one termination criteria that has been.

Optimal Allocation Mechanisms with Single ... - Semantic Scholar
Oct 18, 2010 - We study revenue-maximizing allocation mechanisms for multiple heterogeneous objects when buyers care about the entire ..... i (ci,cLi)], where (pLi)z denotes the probability assigned to allocation z by pLi. The timing is as follows: S

Minimax Optimal Algorithms for Unconstrained ... - Semantic Scholar
Jacob Abernethy∗. Computer Science and Engineering .... template for (and strongly motivated by) several online learning settings, and the results we develop ...... Online convex programming and generalized infinitesimal gradient ascent. In.

Optimal Dynamic Hedging of Cliquets - Semantic Scholar
May 1, 2008 - some presumed mid price of vanillas results in a replicating strategy for the exotic. Risk management departments (also called Risk Control) are charged with feeding the pricing model sensitivity outputs into a VaR model, and generally

TIME OPTIMAL TRAJECTORY GENERATION FOR ... - Semantic Scholar
Aug 13, 2008 - In the near future with the increasing automation and development in ...... Once the point cloud information is available it can be translated into ...

TIME OPTIMAL TRAJECTORY GENERATION FOR ... - Semantic Scholar
Aug 13, 2008 - I would like to thank my committee members Dr.V.Krovi and. Dr.T.Singh ..... points and go to zero at the boundary of the obstacle. In Ref. .... entire configuration space.thus, to satisfy 3.14b the trajectory generated after meeting.

Extracting the Optimal Dimensionality for Local ... - Semantic Scholar
Oct 22, 2008 - 39.2. 3.9. 72.1. 2D LTDA. 52.1. 3.6. 682.5. 51.2. 2.9. 193.7. 38.1. 4.1. 521.8. On this dataset, we randomly select 20 samples per class for ...

The Planning Solution in a Textbook Model of ... - Semantic Scholar
Feb 23, 2004 - This note uses recursive methods to provide a simple characterization of the planner's solution of the continuous time and discrete time version of the simplest Pissarides (2000) model. I show that the solutions are virtually identical

Birth seasonality and calf mortality in a large ... - Semantic Scholar
Jul 18, 2013 - Statistical analyses. Birth seasonality. To test for an effect of month on probability of birth, we implemented a logistic regression model (N = 2350 ele- phants). Each row in the data set was replicated ..... (H. S. M., K. U. M., and

An active feedback framework for image retrieval - Semantic Scholar
Dec 15, 2007 - coupled and can be jointly optimized. ..... space when user searches for cars images. ..... Optimizing search engines using click through data.

On numerical simulation of high-speed CCD ... - Semantic Scholar
have a smaller photosensitive area that cause higher photon shot noise, higher ... that there are i interactions per pixel and PI is the number of interacting photons. .... to the random trapping and emission of mobile charge carriers resulting in ..

Cross-layer Optimal Decision Policies for Spatial ... - Semantic Scholar
Diversity Forwarding in Wireless Ad Hoc Networks. Jing Ai ... One of the practical advantages ... advantage of inducing a short decision delay, the main.