1

Open-loop control of stochastic fluid systems and applications. Bruno Gaujal, Lab. ID, INRIA-IMAG-CNRS-UJF, 38330 Montbonnot, France.

Landy Rabehasaina

1

ENSSAT, 22305 Lannion, France.

keywords: Multimodularity; Fluid queues; Capital allocation.

Abstract: We consider a stochastic process Q(t) satisfying a linear differential equation, driven by a jump process which is modulated by a binary sequence. We prove multimodular properties related to the process Q(t) and apply those results to optimal strategies in storage systems models and in ruin problems.

1

Introduction

The notion of multimodularity goes back to the paper by Hajek [9] who used it for optimal admission control in queues with no information. Other applications in the control of queues with full state information have been given by Weber and Stidham [18] and Glasserman and Yao [8] who proved that there exist optimal policies of threshold type when the cost is multimodular. In [4] Hajek’s result is extended to a general framework. The main results (Theorem 6 p.25 in [4]), used in the present paper, says that under i.i.d. or mere stationary assumptions on the arrival stream and on the service distribution, the optimal open-loop admission control in a queue under the constraint that at most a proportion p of packets may be lost is a bracket sequence. We recall that the admission control can be seen as an infinite binary sequence (ai )i∈N , where ai = 1 means that the ith arriving packet is accepted and joins the queue and ai = 0 means that the ith packet is rejected. Under this notation, the bracket sequence is of the form ai = bp(i + 1) + θc − bpi + θc, where θ is some real number in [0, 1]. The main ingredient in the proof of this result is the multimodularity of the queue size with respect to the control actions. A function f : Zn → R is multimodular with respect to a finite set F ⊂ Zn if for all v, w ∈ F, v 6= w, and all x ∈ Zn , f (x + v) + f (x + w) ≥ f (x) + f (x + v + w). As shown in [9, 2], multimodularity is very close to convexity. This notion has been used in many contexts using various sets F (see for example [18, 12, 3]). The most common set F is the set of right shifts F = {(1, 0, . . . , 0), (1, −1, 0, . . . , 0), (0, 1, −1, 0 . . . , 0), · · · , (0, · · · , 0, 1, −1), (0, . . . , 0, −1)}, used in the present paper. A second important result (Theorem 8 p.27 and Theorems 25 and 26 pp.115–116 in [4]) is for optimal routing to several queues. The optimal policy is again bracket, but the parameter p is unknown is general [7]. For the first time in this paper, this framework is used for stochastic fluid systems. A fluid queue is a storage system where input is modelled by fluid, which is processed out of the system at some rate depending on the state of queue and other external factors (for example a Markov process). This modelling is very common in highspeed telecommunication networks and production systems. The seminal reference for these models is Anick et al. [5], also see Prabhu [14], but other more recent papers like Asmussen and Kella [6], Kella and Stadje [10], feature a dependence on the queue size in the service rate, which is the case here. We show that the cost function is multimodular with respect to the control actions so that the optimal policy is again bracket. The first application concerns admission in a fluid queue. The second application is kind of dual: we show that some functionals of the ruin probability in a portfolio problem are minimized by bracket investment policies. 1 Corresponding author. ENSSAT, 6, rue de Kerampont B.P.805, 22305 Lannion, France. E-mail address: [email protected]

2

1.1

Notations and background.

Let {Nt (a), t ≥ 0} be a non decreasing piecewise constant process with jumps at times {Tn , n ∈ N} such that T0 = 0, {δn = Tn+1 − Tn , n ≥ 1} is a stationary ergodic sequence of random variables, and a = {an , n ≥ 1} is a sequence of integer values numbers. The jumps occur at times {Tn , n ≥ 1}, are modulated by the sequence a = {an }, and the height of the jump is κ(n)

dNt (a) =

X

σk

(1)

k=κ(n−1)+1

Pi at time t = Tn , n ≥ 1, where κ(i) = j=1 ai and {σn , n ≥ 1} is a stationary sequence. We suppose that {σn , n ≥ 1} and {δn , n ≥ 1} are independent. We then consider a stochastic process {Qt (a), t ≥ 0} satisfying the following differential equation  dQt (a) = dNt (a) − µQt (a)dt (2) Q0 (a) = 0 where µ > 0. Qt (a) is a non-negative shot-noise process that can be interpreted for example as the level of a fluid queue. In our context inputs are packets of fluid of sizes σn , coming in batches of an . The queue is served at rate µQt (a), depending on the queue size. Furthermore, we have the following expression for Qt (a) (see e.g. [15]): Z t Qt (a) = exp(−µ(t − s))dNs (a). (3) 0

Besides, it is standard to assume that the sequences {δn = Tn+1 − Tn }, {Tn }, {σn } and a = {an } are double sided (indexed by Z) as they can always be extended to such, so that we may consider queue sizes starting from a negative initial time. Let Θ be a uniformly distributed random variable on (0, 1), independent from {σn } and {δn }. We introduce the randomized bracket sequence {un (Θ, p)} by un (Θ, p) = bp(n + 1) + Θc − bpn + Θc. Finally, we define the n dimensional vectors si = (0, ..., 0, +1 , −1 , 0, ..., 0), i = 1, ..., n − 1, e1 = |{z} |{z} (i−1)th ith (1, 0, ..., 0) and en = (0, ..., 0, 1). Thus F = {e1 , −s1 , ..., −sn−1 , −en }. The paper is organized as follows. We start by giving results on the multimodularity of the expectation of functional of {Q(t)} in Section 2. We then give applications of our results in Section 3.

2

Multimodularity.

If a = {an , n ∈ N} then for all functionals h, E(h(QTn (a))) only depends on a1 , ..., an , which is why we may then write E(h(QTn (a1 , ..., an ))) instead of E(h(QTn (a))). We likewise write Nt (a1 , ..., an ) when t ∈ [0, Tn ]. We have the following lemma. Lemma 1 For all non decreasing convex functions h, an = (a1 , ..., an ) 7→ E(h(QTn (a1 , ..., an ))) is multimodular with respect to F. Proof. We let f (an ) = f (a1 , ..., an ) = E(h(QTn (a1 , ..., an ))). We have to prove that for all u and v in F we have f (an + u + v) + f (an ) ≤ f (an + u) + f (an + v) whenever an + u + v, an + u and an + v has non negative components. Following Altman et al. [4], we construct a coupling associated to the sequence {σn , n ∈ N}. The construction is recalled here for the ˜ t (a1 , ..., an ) = Q ˜ t (a ) the queue size at time t ∈ [0, Tn ] under this sake of completeness. We denote by Q n coupling.

3 • We define the jumps of Nt (an + e1 ) by dNt (an + e1 ) = the jumps are the same as dNt (an )), and by

Pκ(j) k=κ(j−1)+1

σk at times Tj , j = 2, ..., n (i.e.

κ(1)

dNt (an + e1 ) = σ +

X

σk

k=1

at time T1 , where σ a random variable distributed as the σn ’s and such that {σ, σ1 , σ2 , ...} has the same distribution as {σ1 , σ2 , σ3 , ...}. • The jumps of Nt (an − sj+1 ), j = 1, ..., n − 1 at times Ti , i 6= j, are not changed. The jump at time Tj is κ(j)−1 X dNt (an − sj+1 ) = σk k=κ(j−1)+1

and the jump at time Tj+1 is κ(j+1)

dNt (an − sj+1 ) =

X

σk .

k=κ(j)

• The jumps of Nt (an − en ) at times Ti , i 6= n, are not changed. The jump at time Tn becomes κ(n)−1

dNt (an − en ) =

X

σk .

k=κ(n−1)+1

This can be written in the following way: Nt (an + e1 ) = Nt (an − sj+1 ) = Nt (an − en ) =

Nt (an ) + σ 1{t≥T1 } Nt (an ) − σκ(j) 1{t≥Tj } + σκ(j) 1{t≥Tj+1 } Nt (an ) − σκ(n+1) 1{t≥Tn } .

Now let Ytp (p = 1, ..., n − 1), Zt1 and Ztn satisfy the following linear differential equations:  dYtp = σκ(p) δTp (dt) − σκ(p) δTp+1 (dt) − µYtp dt Y0p = 0  dZt1 = σδT1 (dt) − µZt1 dt Z01 = 0  dZtn = σκ(n) δTn (dt) − µZtn dt Z0n = 0

(4)

(5) (6) (7)

where δT is the Dirac measure concentrated at T . The differential equations (2), (5), (6) and (7) are all linear. Consequently, because of expressions (4) we ˜ t (a −si+1 ), Q ˜ t (a −si+1 −sj+1 ), have that for all i and j in {1, ..., n−1} and t ∈ [0, Tn ], the queue sizes Q n n ˜ t (a + e1 ), Q ˜ t (a − en ), Q ˜ t (a + e1 − si+1 ), Q ˜ t (a − en − si+1 ) and Q ˜ t (a + e1 − en ) respectively verify Q n n n n n the following equalities ˜ t (a − si+1 ) = Q n ˜ Qt (an − si+1 − sj+1 ) = ˜ t (a + e1 ) = Q n ˜ t (a − en ) = Q n ˜ t (a + e1 − si+1 ) = Q n ˜ Qt (an − en − si+1 ) = ˜ t (a + e1 − en ) = Q n

˜ t (a ) − Yti+1 Q n ˜ t (a ) − Yti+1 − Ytj+1 Q n ˜ t (a ) + Zt1 Q n ˜ t (a ) − Z n Q t n ˜ t (a ) + Zt1 − Yti+1 Q n ˜ t (a ) − Z n − Yti+1 Q t n ˜ t (a ) + Zt1 − Ztn . Q n

Now from (8) we easily get for all u and v in F and t ∈ [0, Tn ] the following equality ˜ t (a ) + Q ˜ t (a + u + v) = Q ˜ t (a + u) + Q ˜ t (a + v). Q n n n n

(8)

4 If h is convex and non decreasing, by Schur convexity [13], we then get that ˜ t (a )) + h(Q ˜ t (a + u + v)) ≤ h(Q ˜ t (a + u)) + h(Q ˜ t (a + v)). h(Q n n n n

(9)

˜ t (a ) have the same distribution for all vector Since {σn , n ∈ N} is stationary we have that Qt (an ) and Q n an , thus taking expectations in (9) we finally get, at t = Tn , E(h(QTn (an ))) + E(h(QTn (an + u + v))) ≤ E(h(QTn (an + u))) + E(h(QTn (an + v))), i.e. an 7→ E(h(QTn (an ))) is multimodular.

2

Proposition 1 For all n, an = (a1 , ..., an ) 7→ E(h(QTn (an ))) verifies (i) an = (a1 , ..., an ) 7→ E(h(QTn (an ))) is non decreasing in each ai . (ii) if n < m, E(h(QTn (am−n+1 , ..., am ))) ≤ E(h(QTm (a1 , ..., am ))). (iii) if n < m, E(h(QTn (a1 , ..., an ))) = E(h(QTm (0, ..., 0, a1 , ..., an ))). (iv) an = (a1 , ..., an ) 7→ E(h(QTn (an ))) is multimodular. Proof. (iv) was already shown in Lemma 1. For proving (i), we first notice that the jumps of Nt (a1 , ..., aj + 1, ..., an ) are the same as those of Nt (a1 , ..., an ) up to time Tj , which is where we have dNt (a1 , ..., aj + 1, ..., an ) = dNt (a1 , ..., an ) + σκ(j)+1 at time Tj , which means that the jump is increased by σκ(j)+1 . Thus QTj (a1 , ..., aj + 1, ..., an ) ≥ QTj (a1 , ..., an ). The jumps at times Tl , l = j + 1, ..., n of Nt (a1 , ..., aj + 1, ..., an ) have then the same distribution as those of Nt (a1 , ..., an ) and are independent from the state of the queues it follows that P (QTn (a1 , ..., aj + 1, ..., an ) > u) ≥ P (QTn (a1 , ..., an ) > u) for all u ≥ 0. This is equivalent to E(h(QTn (a1 , ..., aj + 1, ..., an ))) ≥ E(h(QTn (a1 , ..., an ))) for all non decreasing function h (see [16]), in particular when h is non-decreasing and convex. This proves (i). We now prove (ii) and (iii) together. Qt (0, ..., 0, a1 , ..., an ) verifies for t ∈ [0, Tm ] Qt (0, ..., 0, a1 , ..., an ) = 0 | {z } m−n

when t ∈ [0, Tm−n ], and dQt (0, ..., 0, a1 , ..., an ) = dNt (0, ..., 0, a1 , ..., an ) − µQt (0, ..., 0, a1 , ..., an )dt

(10)

when t ∈ [Tm−n , Tm ]. Now process Nt (a) satisfies D

dNt (0, ..., 0, a1 , ..., an ) =

˜t−T dN m−n (a1 , ..., an )

Pκ(j) ˜t−T where N m−n (a1 , ..., an ) has jumps of sizes k=κ(j−1)+1 σk at times Tj − Tm−n , j = m − n + 1, ..., m. ˜ Nt (a1 , ..., an ) has the same distribution on [0, Tm − Tm−n ] as Nt (a1 , ..., an ) on [0, Tn ]. This is due to {δn } being stationary and independent from {σn }. Now since Qt (a1 , ..., an ) = 0 at time 0 and verifies dQt (a1 , ..., an ) = dNt (a1 , ..., an ) − µQt (a1 , ..., an )dt D

on [0, Tn ] we conclude that QTn (a1 , ..., an ) = QTm (0, ..., 0, a1 , ..., an ). This yields (iii). (ii) is a consequence since E(h(QTn (am−n+1 , ..., am )) = E(h(QTm (0, ..., 0, am−n+1 , ..., am ))) and since (by (i)) E(h(QTm (0, ..., 0, am−n+1 , ..., am ))) ≤ E(h(QTm (a1 , ..., am ))). 2 We then have the following results:

5 Theorem 1 (a) For all sequence {an , n ∈ Z} of stationary random variables, integer valued and independent from {δn , n ∈ Z} and {σn , n ∈ Z}, QTn (a) converges in distribution as n goes to infinity to a random variable W (a) satisfying Z 0 W (a) = exp(µs)dNs (a). −∞

(b) Let p ∈ [0, 1]. The bracket sequence ai = ui (Θ, p) = bp(i + 1) + Θc − bpi + Θc minimizes E(h(W (a))) for all non decreasing convex function h, among all sequences of stationary random variables {an , n ∈ Z} satisfying    ai ∈ {0, 1}, N 1 X (11) an ≥ p lim inf   N →∞ N n=1

Let us note that the expression or characterization of W (a) in (a) is very standard and can be found for various shot-noise processes (see e.g. Theorem 1 in [11] for a multidimensional process, or expression (3.13) in [6]). Proof. We first prove (a). Since {an , n ∈ Z} and {δn , n ∈ Z} are stationary we have that D

Q0,Tn (a) = QT0 ,Tn (a) = QT−n ,T0 (a) = QT−n ,0 (a) where {Qu,v (a), v ≥ u} (u ∈ R) is defined by  Qu,u (a) = 0 dQu,v (a) = dNv (a) − µQu,v (a)dv, Z

v ≥ u.

0

Now by (3), we have that QT−n ,0 (a) = exp(µs)dNs (a). Since T−n → −∞ as n → ∞, we then have −T−n Z 0 D QTn (a) −→ W (a) = exp(µs)dNs (a) as n → ∞. −∞

We now prove (b). A consequence of Theorem 6 of Altman et al. [4] and Proposition 1 is that, for all N 1 X non decreasing convex function h, the quantity lim sup E(h(QTn (a))) is minimized by a bracket N →∞ N n=1 sequence ai = ui (θ, p) = bp(i + 1) + θc − bpi + θc, for any θ ∈ [0, 1]. In particular, ai = ui (Θ, p) = bp(i + 1) + Θc − bpi + Θc is an optimal stationary sequence. Now, for this stationary sequence, the limsup N 1 X of E(h(QTn (a))) actually happens to be a limit, equal to E(h(W (a))), according to (a). This N n=1 concludes the proof. 2 Let us note that the optimal sequence a defined by an = un (Θ, p) = bp(n + 1) + Θc − bpn + Θc satisfies N 1 X lim an = p. Moreover, one can check by direct computation that E(an ) = p for all n. N →∞ N n=1 Let µ1 and µ2 be positive numbers. Let {an }, an = 0, 1, be a stationary sequence independent from {σn } and {δn }. To avoid tedious technicalities which are pointless for applications, we suppose here, and until the end of this section, that {σn } is i.i.d. Let Q1t (a) and Q2t (1 − a) be the two processes defined by the differential equations  dQ1t (a) = dNt1 (a) − µ1 Q1t (a)dt Q10 (a) = 0  dQ2t (1 − a) = dNt2 (1 − a) − µ2 Q2t (1 − a)dt Q20 (1 − a) = 0 where dNt1 (a) = an σn and dNt2 (1 − a) = (1 − an )σn at time Tn . This means that either Nt1 or Nt2 jump at times Tn , according to the sequence {an }. Here, 1 − a refers to the sequence defined by 1 − an for

6 all n. From Theorem 1 (a), Q1Tn (a) and Q2Tn (1 − a) respectively converge in distribution to W 1 (a) = Z 0 Z 0 exp(µ1 s)dNs1 (a) and W 2 (1 − a) = exp(µ2 s)dNs2 (1 − a). −∞

−∞

The following theorem addresses yet another optimization problem.

Theorem 2 Let H : R2 −→ R be linear and non decreasing in each variable. Let h1 and h2 be convex non decreasing functions. There exists p∗ ∈ [0, 1] such that a sequence of stationary

random variables {an , n ∈ Z} satisfying ai ∈ {0, 1} and minimizing E H h1 (W 1 (a)), h1 (W 2 (1 − a)) is ai = ui (Θ, p∗ ) = bp∗ (i + 1) + Θc − bp∗ i + Θc. Proof. We prove the result in a similar way to Theorem 1 (b). From Theorems 25 and 26 pp.115–116 and Section 6.2 of [4] as well as Proposition 1, there exists p∗ ∈ [0, 1] such that lim inf N →∞

N
1 X E H(h1 (Q1Tn (a)), h2 (Q2Tn (1 − a))) N n=1

is minimum for the stationary bracket sequence {an } defined by an = un (Θ, p∗ ) = bp∗ (n + 1) + Θc − N
1 X E H(h1 (Q1Tn (a)), h2 (Q2Tn (1 − a))) is actually bp∗ n + Θc. Now, as in Theorem 1 (b), the liminf of N

n=1 1 2 a limit equal to E H h1 (W (a)), h1 (W (1 − a)) . 2 Note that Theorem 2 gives the structure of the optimal sequence: the value of the number p∗ is not known in the general case. The value of p∗ is computed numerically in comparable admission control related problems (See Gaujal et al. [7]).

3

Applications

We present two main applications. The first is a generalization of the results of Altman et al. [1, 2] to admission control in the context of fluid queues. Finally, we consider ruin related problems.

3.1

Minimizing a fluid queue with linear service rate in stationary regime.

The first obvious application of the previous results deals with the fluid queue model described in Section 1.1. Let us remember that when Qt (a) is the level in a fluid queue, the service rate is equal to µQt (a). Also recall that fluid comes in packets according to the policy an ∈ {0, 1}. an = 0 means that the nth incoming packet of fluid is rejected, while an = 1 means that it is admitted in the queue. The size of the nth accepted packets is σn . From Theorem 1 (a), QTn (a) converges in distribution to the random variable W (a) as n goes to infinity, when the sequence {an } is stationary. We then have the following result. Proposition 2 Let p ∈ [0, 1]. Suppose that arrivals are Poisson, i.e. that {δn } is an i.i.d. sequence, each δn having the same exponential distribution. Then W (a) is the limiting distribution of the queue. A staN 1 X an ≥ p and minimizing tionary sequence {an } independent from {σn } and {δn }, satisfying lim inf N →∞ N n=1 E(h(W (a))) for all non decreasing convex function h is defined by an = un (Θ, p). Proof. Arrivals being Poisson, and from the PASTA property , the limit in distribution of Qt (a) as t → ∞ is the same as the limit in distribution of QTn (a) as n → ∞, which is W (a). Thus from Theorem 1 (b), an = un (Θ, p) = bp(n + 1) + Θc − bpn + Θc minimizes E(h(W (a))). 2

3.2

Capital allocation problems and blind policies.

We now show applications related to capital allocation problems. The first can be seen as an introductory problem and lays the basis for the second one.

7 3.2.1

A first ruin problem.

Let {Rt (a), t ≥ 0} satisfy the following differential equation  R0 (a) = u > 0, dRt (a) = −dNt (a) + µRt (a)dt.

(12)

Rt (a) can be seen as the capital of a company, continuously reinvested with interest rate µ > 0. It evolves due to outgoing random payoffs at times Ti , that can be occasionally paid for by an independent capital. The payoffs paid by the company are modelled by a stationary sequence {σn }. a = {an } is the sequence that models the policy, interfering in the dynamics of Rt (a). Nt (a) is the amount of money for claims by the company up to time t. It is the process described by (1) and is interpreted easily by the fact that the independent capital can be used at times Ti to pay for the payoffs, thus limiting risks of crash for the capital Rt (a) of the company. The independent capital is used at time Ti only if ai = 0, and is not used if ai = 1. However, a restriction is set on {an } which is that the asymptotic proportion of number of times the company calls on that independent capital must be less than 1 − p. We define the ruin time of the company τ (a) := inf{t ≥ 0| Rt (a) < 0}, with the convention that τ (a) = +∞ if Rt (a) > 0 for all t ≥ 0. We set Pu (τ (a) < +∞) = P (τ (a) < +∞| R0 (a) = u) the probability of ruin starting from the initial capital u. We then consider the following problem:  Z ∞   Pg(u) (τ (a) < +∞)du   minimize 0

N 1 X   an ≥ p, subject to lim inf   N →∞ N n=1

(13)

where g : [0, +∞) −→ [0, +∞) is any continuous, concave and increasing function. Problem (13) is about minimizing an average of the ruin probabilities with respect to the initial surplus. We call {an } a “blind” policy in the sense that {an } is chosen in advance, and does not depend at each time Ti on the state of the process Rt (a) on [0, Ti ]. When the concave function g is for instance defined by g(u) = ln(1 + u)/β for a β > 0, then the cost function to minimize in (13) is exponentially weighted: Z ∞ Z ∞ Pg(u) (τ (a) < +∞)du = Pv (τ (a) < +∞) exp(βv)dv. 0

0

The result of this section is the following: Theorem 3 {an , n ∈ N} defined by ai = u−i (Θ, p) = bp(−i + 1) + Θc − b−pi + Θc, where Θ has a uniform distribution on [0, 1] and is independent from the {Tk+1 − Tk } and the {σk }, solves (13) for all increasing continuous concave function g. Proof. Remembering that a∗n = a−n , and using the same notations as in the proof of Theorem 1 (c), we have that [τ (a) ≤ N ] = [Q−N,0 (a∗ ) ≥ u] for all N ≥ 0 by a standard duality argument (see Asmussen and Kella [6], Sigman and Ryan [17]). This implies Pu (τ (a) ≤ N ) = P (Q−N,0 (a∗ ) ≥ u). (14) Z 0 We know that Q−N,0 (a∗ ) converges almost surely to W (a∗ ) = exp(µs)dNs (a∗ ) as N → ∞. Thus, −∞

by letting N → ∞ in (14), we get the desired equality Pu (τ (a) < +∞) = P (W (a∗ ) ≥ u). We now consider the integral to minimize in Problem (13). It can be easily transformed in the following way: Z ∞ Z ∞ Pg(u) (τ (a) < +∞)du = P (W (a∗ ) ≥ g(u))du 0 0 Z ∞ = P (g −1 (W (a∗ )) ≥ u)du 0
= E g −1 (W (a∗ )) .
But g −1 is increasing and convex. Thus by Theorem 1 (c) the sequence a such that E g −1 (W (a∗ )) is minimum is such that a∗i = a−i = bp(i + 1) + Θc − bpi + Θc. The result follows. 2

8 3.2.2

Capital allocation and ruin problems.

We now address a fairly different problem. Let us now suppose that the company invests its capital in two assets according to a stationary sequence {an }, with an ∈ {0, 1}. We suppose here that the payoffs {σn } form an i.i.d. sequence. The amount of money invested in the first asset is Rt (a). The process {Rt (a), t ≥ 0} satisfies the differential equation  R0 (a) = r0 > 0, (15) dRt (a) = −dNt1 (a) + µRt (a)dt. The amount of money invested in the second asset is St (1 − a). The process {St (1 − a), t ≥ 0} satisfies the following differential equation:  S0 (1 − a) = s0 > 0, (16) dSt (1 − a) = −dNt2 (1 − a) + ηSt (1 − a)dt where η > 0. Nt1 (a) and Nt2 (1 − a) in (15) and (16) are such that dNt1 (a) = an σn and dNt2 (1 − a) = (1 − an )σn at time Tn . The sequence {an } is then such that an = 1 if the nth outgoing payoff σn is paid with the first asset (the one modelled by Rt (a)) whereas an = 0 if the nth outgoing payoff is paid with the second asset (the one modelled by St (1 − a)). Similarly to the previous case, we are interested in finding the optimal policy {an } minimizing both ruin probabilities of Rt (a) and St (1 − a). We thus consider the following cost function: Z ∞   PgR (u) (τ (a) < +∞) + PgS (u) (ξ(1 − a) < +∞) du (17) 0

where τ (a) and ξ(1 − a) are the ruin times of Rt (a) and St (1 − a). gR and gS are concave non decreasing functions from [0, ∞) to [0, ∞). An illustrating example is the simple case where gR (u) = αu, gS (u) = (1 − α)u, where α ∈ (0, 1). gR (u) + gS (u) = u then represents the initial capital of the company; α and 1 − α are the fraction of that initial reserve shared among the two assets. Note that in that case the cost function (17) becomes the weighted averaged ruin probabilities Z ∞  Z ∞  1 1 Pv (τ (a) < +∞)dv + Pv (ξ(1 − a) < +∞)dv . (18) α 0 1−α 0 Thus, when for example α → 0+ then less money is put in Rt than in St as an initial reserve, and in (18) Z ∞ Pv (τ (a) < +∞)dv becomes more important to minimize, as its weight 1/α becomes great. 0

The following theorem gives the optimal policy minimizing (17).

Theorem 4 (a) There exists p∗ ∈ [0, 1] such that an optimal stationary sequence {an } independent from {σn }, {δn } minimizing (17) is defined by an = u−n (Θ, p∗ ). N 1 X an = p, independent N →∞ N n=1 from {σn }, {δn }, and minimizing (17), is defined by an = u−n (Θ, p).

(b) Let p ∈ [0, 1]. An optimal stationary sequence {an } satisfying lim

Note that the interest of point (a) is that the optimal sequence among all policies is a bracket sequence. The downside is that the value of p∗ in the bracket sequence is not explicit except in very few cases. Proof. Using the same duality argument as in the proof of Theorem 3, quantity (17) is equal to
−1 E gR (WR (a∗ )) + gS−1 (WS (1 − a∗ )) (19) Z where WR (a∗ ) =

0

−∞

Z exp(µs)dNs1 (a∗ ) and WS (1 − a∗ ) =

0

−∞

−1 exp(ηs)dNs2 (1 − a∗ ). h1 = gR and

−1 h2 = gS−1 are convex and non decreasing. Theorem 2 with H : (x, y) 7→ x + y, h1 = gR and h2 = gS−1 yields the results for point (a). We now prove point (b). Let p ∈ [0, 1] be fixed. For any fixed stationary sequence b = {bn }, the staN
1 X −1 −1 ∗ ∗ an = p tionary sequence {an } minimizing E gR (WR (a )) + gS (WS (b )) and verifying lim N →∞ N n=1

9 is defined by an = u−n (Θ, p). This comes from Theorem 1 and from the fact that h1 is convex and non decreasing. Having fixed an = u−n (Θ, p) for all n, a stationary sequence {bn } minimizN
1 X −1 (WR (a∗ )) + gS−1 (WS (b∗ )) and verifying lim ing E gR bn = 1 − p turns out to be bn = 1 − N →∞ N n=1 u−n (Θ, p) = 1 − an , still from Theorem 1 and the fact that h2 is convex and non decreasing. 2

References [1] E. Altman, B. Gaujal, and A. Hordijk. Admission control in stochastic event graphs. IEEE Transaction on Automatic Control, 45(5):854–868, 2000. [2] E. Altman, B. Gaujal, and A. Hordijk. Multimodularity, convexity and optimization properties. Mathematics of Operations Research, 25(2):324–347, 2000. [3] E. Altman, B. Gaujal, and A. Hordijk. Regular ordering and applications in control policies. Journal of Discrete Event Dynamic Systems, 12(2):187–210, 2002. [4] E. Altman, B. Gaujal, and A. Hordijk. Discrete-Event Control of Stochastic Networks: Multimodularity and Regularity. Number 1829 in LNM. Springer-Verlag, 2003. available at http://wwwsop.inria.fr/maestro/personnel/Eitan.Altman/PAPERS/bookmm.pdf. [5] D. Anick, D. Mitra, and M. M. Sondhi. Stochastic theory of a data-handling system with multiple sources. Bell System Tech. Journal, pages 1871–1894, 1982. [6] S. Asmussen and O. Kella. Rate modulation in dams and ruin problems. Journal of Applied Probability, 33:523–535, 1996. [7] B. Gaujal, E. Hyon, and A. Jean-Marie. Optimal routing in two parallel queues with exponential service times. Journal of Discrete Event Dynamic Systems, 2005. To appear in the special issue of selected papers from WODES. [8] P. Glasserman and D. D. Yao. Monotone optimal control of permutable GSMPs. Mathematics of Operations Research, 19:449–476, 1994. [9] B. Hajek. Extremal splittings of point processes. Mathematics of Operations Research, 10(4):543–556, 1985. [10] O. Kella and W. Stadje. Exact results for a fluid model with state-dependent flow rates. Probability in the engineering sciences, 16(4):389–402, 2002. [11] O.Kella and W.Stadje. Markov modulated linear fluid networks with Markov additive input. Journal of Applied Probability, 39(2):413–420, 2002. [12] G. Koole. Structural results for the control of queueing systems using event-based dynamic programming. Technical Report WS-461, Vrije Universiteit Amsterdam, 1996. [13] A. W. Marshall and I. Olkin. Inequalities: Theory of Majorization and its Applications, volume 143 of Mathematics in Science and Engineering. Academic Press, 1979. [14] N. U. Prabhu. Stochastic storage processes. Queues, insurance risk, dams and data communication. Springer, 1997. [15] D.Revuz and M.Yor. Continuous martingales and brownian motion. Springer, 1999. [16] M. Shaked and J.G. Shantikumar. Stochastic orders and their applications. Academic press, 1994. [17] K. Sigman and R. Ryan. Continuous-time stochastic recursions and duality. Advances in Applied Probability, 32:426–445, 2000. [18] R.R. Weber and S. Stidham. Optimal control of service rates in networks of queues. Advances in Applied Probability, 19:202–218, 1987.