Production Priorities in Dynamic Relationships Jean Guillaume Forand†

∗

Jan Z´apal‡

March 23, 2018

Abstract We characterise optimal contracts in a dynamic principal-agent model of joint production, in which heterogenous project opportunities arrive stochastically and are publicly observed upon arrival, utility from these projects is non-transferable and the agent has the option to quit the relationship at any time. We show that the principal demands projects early in the relationship (production that benefits the principal but not the agent) and supplies projects late (production that benefits the agent but not the principal), and that the production of projects is ordered by their cost-effectiveness, that is, by their associated efficiency in extracting (for demanded projects) and providing (for supplied projects) utility to the agent.

JEL Classification: C73; D86; L24 Keywords: Dynamic Contracts; Incentive Provision; Heterogeneous Projects

∗

We thank Georgy Egorov, Paul Klein, Colin Stewart, Jakub Steiner, Xin Zhao, the seminar participants at Queen’s, Toronto and Wilfred Laurier, as well as the audiences at the 2015 Annual Conference of the Canadian Economic Association, the 2016 Annual Meeting of the Midwest Political Science Association, the 2016 Canadian Economic Theory Conference, the 2016 joint meeting of the European Economic Association and Econometric Society, the Fall 2016 Midwest Economic Theory Conference and the 2017 North American Summer Meeting of the Econometric Society. Finally, the Coeditor, Simon Board, and two anonymous referees have provided excellent feedback. This paper was previously circulated under the title ‘The Demand and Supply of Favours in Dynamic Relationships’. J.G. Forand acknowledges support from a SSHRC IDG. † Department of Economics, University of Waterloo, Hagey Hall of Humanities, Waterloo, Ontario, Canada N2L 3G1. Email: [email protected]. Website: http://arts.uwaterloo.ca/∼jgforand. ‡ CERGE-EI, Politickych veznu 7, 111 21 Prague, Czech Republic. Email: [email protected]. Website: https://sites.google.com/site/jzapal.

1

Introduction

Productive relationships generate a variety of joint project opportunities over their lifetimes, so that their output dynamics resolve a non-trivial project selection problem. First, because the parties to the relationship can have conflicting preferences over potential projects, what differentiates the opportunities that are produced from those that are passed over? And second, because the availability and quality of project opportunities change over time, at what stages of the relationship does production occur? In this paper, we address these questions in a dynamic principal-agent model in which (a) heterogenous project opportunities arrive according to an arbitrary stochastic process, (b) utility from these projects is non-transferable (although transferable utility is a special case of our model), and (c) the principal is contractually committed to production decisions but the agent can walk away from the relationship at any time. By (c), the agent must be continually incentivised to remain in the relationship, but by (b) the principal can typically reward current production only through commitments to future production. But then, by (a), the scope, scale and timing of future project opportunities is critical for the power of the incentives at the principal’s disposal. We characterise optimal contracts in our setting and detail the dynamics of the principal’s demand and supply of projects (i.e., the production of projects that benefit the principal but are costly for the agent, and vice-versa). Although we model a canonical principal-agent relationship, for the remainder of the Introduction we fix ideas by focusing on a manager-worker pair within a larger firm. The simplest model of their interaction features two time-invariant project opportunities: at each stage of their relationship, the manager demands effort from the worker and supplies a wage (Mirrlees, 1976). In contrast, our goal is to allow for a rich set of productive activities that arise in the course of this relationship. For example, the projects that the manager can demand from the worker, which can flow in from other units of the firm, need not be of equal quality over time: some are better fits for the skill sets of the manager or the worker; some can arrive in periods in which the manager is being assessed or monitored by her own superiors, or in busy periods in which additional production requires overtime hours from the worker; or more broadly the dynamics of project opportunities within the firm can be driven by the business cycle or industry trends. Similarly, the manager’s discretion in day-to-day activities allows her to supply a variety of projects to the worker. Many of these need not involve monetary payments, especially in cases when most financial terms in the worker’s contract are either set by the firm’s human resources department or regulated by collective bargaining agreements (although even here the manager may have the ability to periodically recommend the worker for bonuses, which our model can accommodate). Non-monetary compensation is known to be important for workers’ motivation and job satisfaction, and its use is both widespread in practice and the subject of 1

extensive research (Milkovich, Newman, and Gerhart, 2013). For example, the manager can supply informal perks like travel opportunities, accommodations for family issues and better office space, and even status goods like firm or industry-specific awards. The manager can also tilt task allocations towards those that are more in line with the worker’s interests, or more beneficial to his prospects for career advancement (e.g., involving training programs). Finally, any dynamics that affects the worker’s skills, the manager’s ability to attract different kinds of projects into their unit, or the growth prospects of the firm will have an impact on the various ways that the worker can be compensated. In the case in which the manager demands effort and supplies money, it is well known (Lazear, 1981) that the manager benefits from delaying the worker’s compensation: whereas current payments are sunk when the manager makes future demands for effort, committing to pay the worker in the future motivates both current and future effort. Not surprisingly, the optimal contracts in our model will also feature backloaded compensation, but because opportunities to reward the worker are heterogenous, time-varying and possibly scarce, the central task is to determine how the manager establishes priorities among the projects that she can supply. If, for example, the worker prefers increased flexibility in his schedule to firm-specific performance awards, then will the principal commit to more schedule flexibility in the future before committing to nominating the worker for awards? The answer, in general, is no, because focusing only on the workers’ preferences neglects the manager’s costs from supplying projects. If awards have value for the worker and are relatively cheap for the manager, then she must commit to supplying all future awards for which the worker is eligible before making any promises about future scheduling flexibility, irrespective of when or how often these opportunities to compensate the worker arise. The effectiveness of awards as a tool for compensating the worker generates gains from trade for the principal across supply projects: if the manager ever commits to future flexibility in the agent’s schedule while some award opportunity is not exploited, she could benefit from intertemporally reallocating commitments away from scheduling flexibility. This intuition underlies our main result characterising optimal contracts: we show that the manager always prioritises her supply of projects according to their cost-effectiveness: their benefit for the worker relative to their cost for the manager. At any point in the relationship, the optimal contract identifies a threshold supply project and projects that are more cost-effective than this threshold are supplied in all their future occurrences regardless of the realisations of the stochastic process. Projects that are less cost-effective than the threshold are not supplied unless (a) the manager makes a new demand and (b) the worker’s participation constraint requires fresh commitments to supply future projects. Because the manager adds new supply commitments through cost-effectiveness, the threshold project transitions to less cost-effective

2

projects over time. How does the growth in the scope of supplied projects affect the manager’s demand for projects? We show that, at any stage of the relationship, the manager demands only those projects that are more cost-effective than the threshold supply project (where costeffectiveness for demand projects measures the benefit for the manager relative to the cost for the worker). Therefore, the manager’s accumulation of increasingly less cost-effective supply commitments is tied to rationing of her demands on the worker, which become concentrated only on the most cost-effective projects. Broadly speaking, the manager makes demands early in the relationship and supplies projects late, and the variety of tasks that the worker accomplishes for the firm shrinks over time while the extent of the firm’s activities in his favour grows. To illustrate our main results further, suppose that the manager can demand that the worker completes high or low-priority projects, with the effort cost being the same for both types of projects but high-priority projects being more important for the firm. Therefore, high-priority projects are more cost-effective. Suppose further that projects of any priority are more costeffective than supplying the worker with awards but that only high-priority projects are more cost-effective than supplying the worker with schedule flexibility. Early in the relationship, the manager demands projects of all priorities and commits to supply awards only.1 While there remains any future award that has not been promised, it is not optimal for the manager to pass up any demand: otherwise, the manager could exploit intertemporal gains from trade by making additional demands for either high or low-priority projects in exchange for promise of awards. Later in the relationship, once the manager has promised all future awards as well as some schedule flexibility, it is not optimal for her to demand low-priority projects: otherwise, the manager could increase her surplus by passing over low-priority demands and clawing back some commitments to future scheduling flexibility. The dynamics of the demand and supply of projects, through which the manager maintains the worker’s incentive to remain with the firm, reflects the inefficiency that stems from the worker’s inability to commit to production decisions. Specifically, the manager would benefit from being able to use past supplies to incentivise current demands. For example, early opportunities to supply awards are passed over before any project has been demanded, but late opportunities to demand low-priority projects are passed over when the manager can only promise flexible scheduling in exchange. If the worker had ex ante commitment power, then the manager could benefit from trading early supplies of awards against late demands for low-priority projects. In fact, as a by-product of our main results, we show that ex ante Pareto-efficient contracts involve a time-invariant threshold supply project. 1

We are illustrating typical dynamics of the optimal contract, the details of which would depend on, among other things, the details of the process driving projects opportunities. We also revisit the current example in Section 4.

3

Cost-effectiveness pins down production priorities, but not the exact dynamics of production decisions. For example, how often does the manager demand low-priority projects, or when does the principal start supplying both schedule flexibility and awards? Answering these questions requires determining the worker’s value from the relationship at any point in time, which sets the level of his participation constraint at that stage of the relationship. This value, which is endogenous, incorporates the worker’s utility from producing projects, his time preferences and the availability of projects determined by the process driving project opportunities: when this process is arbitrary, the relationship value has very little structure. In Section 5, we specialise the model to the case of Markov project processes and construct optimal contracts directly. In doing so, we rank the manager’s demands by how expensive they are for her: more expensive demands require more generous supplies of projects to the worker. We exploit this ranking of demands’ expensiveness to give a complete characterisation of production dynamics. Because we study how future opportunities provide incentives for current production, our work has connections to the literature on informal risk-sharing in the presence of stochastic endowment shocks (Thomas and Worrall, 1988; Kocherlakota, 1996; Dixit, Grossman, and Gul, 2000). Important generalisations of this work incorporate hidden information, about endowment shocks or utility from production, as well as sequential actions. The former literature analyses chips mechanisms (M¨obius, 2001; Hauser and Hopenhayn, 2008) and dynamic contracts with and without commitment (Guo and H¨orner, 2015; Lipnowski and Ramos, 2016). The latter literature studies hold-up situations (Thomas and Worrall, 1994, 2018; Board, 2011) and has close links to the relational contracts literature (Levin, 2003). Furthermore, our work is related to the literature on dynamic principal-agent interactions (Lazear, 1981; Rogerson, 1985; Spear and Srivastava, 1987; Sannikov, 2008). Our focus on heterogeneous projects and their selection for production is the key difference between these contributions and ours. Furthermore, we assume that players are risk-neutral, so that risk-sharing plays no role in our results; we do not rely on transfers, which makes our optimal relationships non-stationary, and unlike most relational contracts; we place no restrictions on the process driving project opportunities, as opposed to the standard iid or Markov assumptions;2 and we abstract from information asymmetries and hold-up problems, which, then, play no role in our model. Three papers are most closely related to ours. First, Ray (2002) shows that any efficient principal-agent relationship backloads the agent’s compensation: by increasing the agent’s continuation value the principal relaxes the agent’s no-deviation constraint, so that she can make the agent work harder and improve efficiency. In our model this logic is one of the forces that 2

From a technical point of view, this rules out standard recursive approaches to characterising optimal dynamic contracts (Spear and Srivastava, 1987; Thomas and Worrall, 1988; Abreu, Pearce, and Stacchetti, 1990). In contrast, our proofs rely on direct arguments.

4

drive the backloading of the agent’s utility: by promising to supply the threshold project in the future, the principal gains the ability to demand projects that are more cost-effective than this threshold. The other reason is the rationing in the principal’s demands stemming from her accumulation of increasingly less cost-effective supply commitments. This latter reason has no analog in Ray (2002), which features a repeated stage game and hence no heterogeneity in future production opportunities. Second, Bird and Frug (2017) study project production in a closely related dynamic principalagent model, in which project arrivals follow independent Poisson processes and are privately observed by the agent. Like us, they highlight the criterion of cost-effectiveness for prioritising project production. Unlike us, they show that the principal frontloads the agent’s compensation, in that she might make less cost-effective supply commitments before exhausting all more cost-effective supply commitments. Informational asymmetries are the key to understanding why our results differ from theirs. In their environment, the principal’s only tool to incentivise the agent to disclose the arrival of a demand project is the growth in the agent’s continuation value. Therefore, frontloading the agent’s compensation allows the principal to free up incentives for future disclosures, and the principal trades off prioritising cost-effective projects against future flexibility. In our model with commonly observed project opportunities, it is the level of the principal’s future commitments that underpins the agent’s incentives to accede to a demand. Consequently, the agent’s compensation is backloaded because the principal always follows cost-effectiveness when making supply commitments, and the (inefficient) variability in the set of projects that the principal demands and supplies eventually vanishes as the optimal contract converges to an ex ante Pareto-efficient one. Third, Samuelson and Stacchetti (2017) study the role of transfers in a version of our model with two-sided lack of commitment and an iid process driving project opportunities. They show that the principal uses variation in either continuation values or transfers to generate incentives when transfers are either absent or present, respectively. Their model, however, does not admit a simple description of the relationship’s dynamics. In our model, we can capture transfers to the agent or to the principal through suitably defined supply and demand projects. Because the principal follows cost-effectiveness when committing to supply projects, our results imply that the principal will not start paying the agent until she has exhausted more cost-effective means to reward him. Moreover, the relationship dynamics in our model initially favours the principal and eventually favours the agent. This implies that, when available, transfers flow towards the principal early in the relationship and towards the agent later in the relationship.

5

2

Model

A principal and an agent participate in a long-lived relationship in which a joint project opportunity arises in each period t = 1, 2, . . .. Specifically, let U ⊂ R2 be a finite set and let u = {ut }t≥1 be a U ∪ {(0, 0)}-valued stochastic process that describes the arrival of projects over time, where ut = (0, 0) denotes the absence of a project at t. Let ut = (u1 , . . . , ut ) denote a project history at t, and let H denote the set of all such histories for all times t. Because optimal contracts are indeterminate at histories that occur with zero probability, we assume that P0 (ut ) > 0 for all project histories ut . This is the only assumption that we impose on the project process u for our main results, and we do so mainly to ease the exposition.3 Given a project ut at time t, the principal and the agent simultaneously decide whether or not to participate in the production of the project, and project ut is produced if and only if both players agree to produce it. We let ut = (uP,t , uA,t ) denoted the payoffs to the principal and the agent if project ut is produced, and we normalise each player’s payoff from no production to 0. For simplicity, we assume that the players’ stage preferences over the production of projects are strict, that is, that uA,t 6= 0 and uP,t 6= 0 for all projects ut ∈ U. Therefore, player i (myopically) prefers to participate in the production of project ut if ui,t > 0 and prefers not to participate if ui,t < 0. We model projects parsimoniously, but we can easily accommodate projects which are more complicated ventures with uncertain outcomes: in this case, ut is interpreted as the expected utilities to the principal and the agent from these richer lotteries. Finally, the players discount future payoffs with common factor δ ∈ (0, 1). Both project histories and production decisions, and hence all players’ payoffs, are publicly observable and verifiable. A contract κ : H → [0, 1] maps project histories into production probabilities. Given a project history ut at time t, κ(ut ), henceforth κt for short with history ut understood, is the probability with which contract κ specifies that the project at t is produced. That is, a contract specifies a complete plan for what projects should be produced by the principal and the agent in all contingencies that can arise during their relationship. Furthermore, contracts allow for the use of a public randomisation device which determines whether or not production occurs following any given history.4 Let K denote the set of all contracts.5 3

Any process with zero-probability events can be expressed as the limit of a sequence of processes without such events. The limit of the corresponding sequence of optimal contracts, as characterised by our results, is an optimal contract for the limiting process. In this sense, our assumption generates a selection of optimal contracts for project processes with zero-probability events. 4 Interior production probabilities are useful to resolve rounding issues associated with the fact that production choices are discrete. As we show below, optimal contracts are essentially bang-bang. 5 Allowing contracts to depend on a richer notion of histories, which record outcomes of the randomisation in production, would not change our results. This randomisation can be used only to make the agent’s continuation utility random, and can be weakly improved upon due to the convexity of the underlying utility possibility set.

6

Given a contract κ and a history ut at time t, let Ui,t = Et

∞ X

0

δ t −t κt0 ui,t0 ,

t0 =t

denote the associated discounted sum of payoffs to player i starting from t. The expectation is taken conditional on the information available at t, which resides in project histories ut , but, as for contracts, we leave the history-dependence of payoffs implicit to lighten notation. Notice that the linearity of the stage utilities in production probabilities implies that intertemporal smoothing of production decisions due to risk-aversion plays no role in our results. We assume that production decisions within the relationship are contractible, but that the agent has the option to irreversibly quit the relationship at the beginning of every period t, after the realisation of project ut but before the realisation of the production decision (determined by κt ). Quitting yields a payoff of 0 to both players, which is the payoff they receive when no project is ever produced. It follows that an optimal contract κ∗ is a solution to the problem max E0 UP,1 κ∈K

subject to UA,t ≥ 0 for all project histories ut .

(IRA,t )

In words, an optimal contract maximises principal’s ex ante utility from the relationship subject to being individually rational for the agent following all project histories.6 In all periods, an optimal contract must specify production decisions that are (stage) Paretoundominated. This implies that if the preferences of the principal and the agent over the project at t are aligned, then an optimal contract implements jointly optimal production decisions.7 Lemma 1. If contract κ∗ is optimal, then 1. if uP,t , uA,t > 0, then κ∗t = 1, and 2. if uP,t , uA,t < 0, then κ∗t = 0. Lemma 1 confirms that optimal contracts can be identified with the production decisions they prescribe for those projects on which the principal and the agent disagree. To this end, define the sets D = {v ∈ U : vP > 0 > vA } and S = {w ∈ U : wA > 0 > wP }. Given a contract κ, we say that the principal demands a project with probability κt at t whenever vt ∈ D, and conversely that the principal supplies a project with probability κt at t whenever wt ∈ S. The 6

By the countability of project histories, standard arguments establish that an optimal contract always exists (e.g., Dixit et al., 2000). 7 The proofs of all results are in the Appendix.

7

decomposition of an optimal contract into the demand and supply of projects, while trivial, turns out to be a fruitful way to describe project selection dynamics.

3

Benchmark: Ex Ante Pareto-Efficiency

A useful benchmark for our model is that of ex ante Pareto-efficient contracts, which allow the agent to commit to production decisions. These contracts maximise the principal’s ex ante utility subject to a lower bound u on the agent’s ex ante utility. Formally, an efficient contract κe is a solution to max E0 UP,1 subject to E0 UA,1 ≥ u. κ∈K

Efficient contracts are straightforward to describe, but they depend on many of the trade-offs that are resolved by optimal contracts (when the agent has the option to walk away from the relationship at any time). Therefore, we introduce and discuss these key properties in this simpler setting, and in Section 4 we detail how they are affected when the agent must be continually incentivised to support production. Define an ordering of projects such that u  u0 if and only |uP/uA | > |u0P/u0A |. In words, if v, v ∈ D and v  v, then project v is more cost-effective to demand than project v for the principal: in this case the ratio vP/|vA |, the principal’s benefit per util cost to the agent, measures the productivity of project v as a tool for extracting utility from the agent. Conversely, if w, w ∈ S and w  w, then project w is more cost-effective to supply than project w for the principal: in this case the ratio |wP |/wA , the principal’s cost per util benefit to the agent, measures the productivity of project w as a tool for providing utility to the agent. This is illustrated in Figure 1, where points in the plain represent projects, projects in the northwestern quadrant can be demanded by the principal, projects in the southeastern quadrant can be supplied, and cost-effective projects are represented by larger dots. For simplicity, we assume that the ordering  is complete on D ∪ S, i.e., that all project pairs are ranked strictly by cost-effectiveness. We now show that the principal’s demand and supply of projects in efficient contracts are determined by cost-effectiveness. Because our characterisation of optimal contracts in Section 4 can be applied directly to characterise efficient contracts, the following result is in fact a corollary of Proposition 2, our main result. Proposition 1. Fix any ex ante Pareto-efficient contract κe and any time t. (i) The principal supplies cost-effective projects.

8

There exists a threshold supply project W e ∈ S such that κet

=

 1

if W e  wt ,

0

if wt  W e .

(1)

(ii) The principal demands projects that are cost-effective relative to those that are supplied.

κet

=

 1

if vt  min {w  W e },

0

if max {W e  w}  vt .

(2)

Figure 1: Ex ante Pareto-efficient contracts uP

D e

κ

= 1

κe

=

v

0

v uA W W We S

Part (i) says that the principal’s supply decisions can be represented by a history-independent threshold project: at any point in the relationship, those supply projects that are more costeffective than W e (these lie above the thick solid line in Figure 1) are produced and those that are less cost-effective than W e (these lie below the thick solid line) are not. What drives this result is that the principal can always reallocate production decisions that do not follow costeffectiveness. If a less cost-effective project W was ever produced following any project history while a more cost-effective project W was passed over following some other history, then the principal could gain by shifting some production probability from W to W while keeping the agent’s ex ante utility fixed: the (expected discounted) benefit that the agent receives through W can be supplied through W at a strictly lower (expected discounted) cost to the principal. Put differently, the principal only commits to supplying less cost-effective projects after all production opportunities with more cost-effective projects have been exhausted. 9

Part (ii) says that the threshold supply project W e also identifies the principal’s demand for projects: loosely speaking, the principal only demands projects that are more cost-effective than W e . What drives this result is a straightforward cost-benefit calculation. More precisely, let the supply project W = min {w  W e } be the most cost-effective project among those that the principal never supplies. It follows that if the principal were to make additional demands on the agent, project W is an upper bound on their corresponding incentive costs. Therefore, the principal cannot pass over any opportunity to demand some project v more cost-effective than W (above the dotted line in Figure 1) without failing to exploit some gains from trade: the principal can gain by increasing production of both v and W while keeping the agent’s ex ante utility fixed. Conversely, let the supply project W = max {W e  w} be the least cost-effective project among those that the principal always supplies. The project W is a lower bound on the savings that the principal could achieve by curtailing her demands, so that any demand v that is less cost-effective than W (below the dashed line in Figure 1) is surplus-destroying for the principal: the principal can gain by decreasing production of both v and W while keeping the agent’s ex ante utility fixed. For ease of exposition, Proposition 1 does not describe production decisions at the threshold project W e . In fact, this obscures an indeterminacy in efficient contracts: the agent’s ex ante utility constraint identifies the total (expected discounted) quantity of production at the threshold project (and hence the principal’s ex ante payoff also), but the linearity of the players’ payoffs in production probabilities allows arbitrary distributions of that production across project histories. In the Appendix, we characterise production decisions at all projects and resolve this indeterminacy by selecting efficient contracts that can be expressed through time thresholds. However, an important note is that the ordering of production decisions through cost-effectiveness implies that all efficient contracts give rise to the same threshold project. Specifically, given two ex ante Pareto-efficient contracts κe and κ0 e , we have that W e = W 0 e . It may seem puzzling that our description of ex ante Pareto-efficient contracts leaves out important factors like the scale of projects (i.e., the absolute values of uP and uA ), the players’ time preferences (i.e., the common discount factor δ) and the dynamics of project opportunities (i.e., the properties of the project process u).8 On the one hand, Proposition 1 states that the principal can establish production priorities among heterogenous projects by relying only on the cost-effectiveness criterion. On the other hand, any factor that hinders the principal’s ability to reward the agent by supplying projects throughout the relationship will restrict her ability to make demands. Put differently, if, for example, all demands that can be made of the agent 8

However, the fact that the discount rate is common is critical for the intertemporal reallocations of production that underly all our arguments.

10

impose a high stage cost on him, if players are impatient or if supplied projects only arrive late in the relationship, then ex ante Pareto-efficient contracts will be more generous towards the agent: the threshold project W e will be less cost-effective which, in turn, means that the principal’s demands will be restricted to a smaller set of more cost-effective projects.

4

Optimal Contracts

We return to our main model in which only the principal is contractually committed to production decisions. The problem solved by optimal contracts has a complex set of interlaced individual rationality constraints for the agent, but much of the insights developed in the study of efficient contracts in Section 3 carry over to this richer setting. In particular, the following characterisation of optimal contracts, which is the main result of our paper, confirms the key role of cost-effectiveness in project selection. Proposition 2. Fix any optimal contract κ∗ and any project history ut−1 . (i) The principal supplies cost-effective projects. ∗ ∈ S such that There exists a threshold supply project Wt−1 κ∗t =

 1

∗  wt , if Wt−1

0

∗ if wt  Wt−1 .

(3)

(ii) The principal demands projects that are cost-effective relative to those that are supplied.

κ∗t

 > 0

∗ }, if vt  min {w  Wt−1

= 0

∗ if max {Wt−1  w}  vt .

(4)

(iii) Over time, the principal increases her supply of projects and decreases her demands. 0 ∗ ∗ If ut−1 is a subhistory of ut −1 , then either Wt∗0 −1 = Wt−1 or Wt∗0 −1  Wt−1 . Parts (i) and (ii) are closely related to their corresponding parts in Proposition 1: part (i) says that the principal’s supply decisions can be described by a threshold project, with projects more cost-effective than this threshold being supplied while less cost-effective ones are not; and part (ii) says that the principal must (a) demand (with positive probability) any project that is more cost-effective than the most cost-effective project that she is not currently ∗ committed to supply (i.e., project min {w  Wt−1 }) and (b) pass over any demand that is less cost-effective than the least cost-effective project that she is currently committed to supply 11

∗ (i.e., project max {Wt−1  w}). This is illustrated in Figure 2, in which the set of produced ∗ projects is contained in the greyed area. Critically, the threshold project Wt−1 is historydependent, whereas the efficient threshold W e was fixed across time. This distinction highlights the role of the agent’s ongoing option to quit the relationship. Specifically, different project histories ut−1 will inherit different sequences of demands by the principal, and incentivising these demands will have required different commitments to supplying projects in the future. ∗ Correspondingly, the threshold Wt−1 tracks the commitments that the principal accumulated t−1 along project history u , and equations (3) and (4) describe how these commitments are tied to production decisions on project ut at history (ut−1 , ut ). Note that the principal’s current supply of projects would necessarily depend on past demands even if, as in the special case in which the process u is iid, the distribution of future production opportunities was the same in all periods.9 For arbitrary project processes, the fact that the relationship’s future productivity depends on the full project history generates an additional, forward-looking, rationale for history-dependent supply commitments. Finally, as was the case for efficient contracts, while there is indeterminacy in optimal production decisions at threshold projects, the principal’s incentive to select projects according to cost-effectiveness ensures the threshold’s uniqueness: given two optimal contracts ∗ κ∗ and κ0 ∗ , we have that Wt−1 = W 0 ∗t−1 for all project histories ut−1 .10

Figure 2: Optimal contracts

κt

=

> 0

0

>0

=0

κt

uP

κ t0

κ t0

D

v

v0 v uA

w

w0

∗ Wt−1

w Wt∗0 −1

S

∗ Not only does the optimal supply threshold Wt−1 vary across project histories at any given time, but part (iii) says that, along any given history, it becomes more generous towards the 9

We will show this explicitly in our construction of optimal contracts when u is Markov, in Section 5. This indeterminacy does not affect the principal’s ex ante payoff, and as for efficient contracts, we resolve contractual indeterminacy in the Appendix by selecting contracts that can be represented by (history-dependent) time thresholds. 10

12

agent over time by transitioning to less cost-effective projects. Combined with part (ii), it follows that the principal’s demands become concentrated on successively smaller sets of more cost-effective project. This is illustrated in Figure 2 in which, given t0 > t, the more generous supply threshold leads to a clockwise shift in the region containing produced projects, which is now the dotted area. The threshold project’s dynamics is driven by two key incentives, one for the principal and one for the agent. First, as for efficient contracts, the principal never supplies any less cost-effective project before all her future opportunities to supply more cost-effective projects have been exhausted: therefore, the threshold project cannot become more cost-effective along any given history. As in the case of efficient contracts, this is established by considering intertemporal reallocations of production decisions that fail to follow cost-effectiveness, although the arguments are more delicate in this case because such reallocations must be constructed within-histories and in such a way that no intervening individual rationality constraints are violated. Second, if, following some demand, the agent’s individual rationality constraint would fail at the current supply threshold, then the principal must commit to supplying more projects in the future, with the additional commitments necessarily less cost-effective than the current threshold. In particular, whenever the threshold moves in the direction of the agent it must be ∗ that a binding individual rationality has been met: if any history ut is such that Wt∗  Wt−1 , ∗ 11 then we have that the principal made a demand at t and that UA,t = 0. ∗ is monotone in cost-effectiveness along any project hisBecause the threshold project Wt−1 tory and there is a finite number of projects, it follows that this threshold is eventually time0 ∗ invariant: if t is large enough and ut−1 is a subhistory of ut −1 , then we have that Wt∗0 −1 = Wt−1 . This immediately implies the following relationship between optimal and efficient contracts. ∗ Corollary 1. Along any project history, the supply threshold Wt−1 associated to any optimal contract converges to the supply threshold associated to some ex ante Pareto-efficient contract.12

The inefficiency of optimal contracts arises from the way in which the agent’s incentives link the principal’s demands to her supply of projects. In efficient contracts, the principal’s total (expected discounted) supply of projects provides the agent with the incentives to commit to respect the principal’s total (expected discounted) demands, but the principal is otherwise free to use her past supply of projects to support current demands. When the agent can quit the 11

The fact that the supply threshold responds to demands is also what explains the difference between equation (2) for efficient contracts and equation (4) for optimal contracts, specifically that any project vt  min {w  ∗ W e } must be produced with probability 1 whereas we can guarantee only that any project vt  min {w  Wt−1 } is produced with positive probability. In the latter case, if project vt is produced with interior probability, then this project is not produced in any project history succeeding (ut−1 , vt ). 12 Note that any w ∈ S is a threshold supply project of an ex ante Pareto-efficient contract with appropriately chosen u ¯ in the optimisation problem defining ex ante Pareto-efficient contracts.

13

relationship at any time, past rewards are sunk and provide no incentives for his current production decisions. Therefore, relative to efficient contracts, the principal undersupplies projects early in the relationship. To illustrate this, note that κ∗t = 0 for all histories (ut−1 , wt ) for which ∗ = min S for such histories), whereas no demand was made in any subhistory of ut−1 (i.e., Wt−1 under an efficient contract a project can always be supplied in the relationship’s initial period (i.e., if W e  w1 ). This temporal linkage, through which the supply of projects trails their demand, leads to production inefficiencies within histories: the principal passes up the opportunity to supply projects early that she supplies later (w0 in Figure 2), and correspondingly fails to makes some later demands that she would make earlier (v 0 in Figure 2), and the agent’s incentives prevent using early unsupplied projects to support later undemanded ones. There are also production inefficiencies across histories. For example, along some histories in which the principal has made many demands or in which the project process is not conducive to rewarding the agent, the principal will not have the ability to demand projects that she could demand following histories that are more favourable to the agent. In such cases, commitment power for the agent would allow the principal to reallocate production across histories and subsidise demands in the former types of histories through supply in the latter types. Example. Returning to the manager-worker application from the Introduction, we can expand on the example that we discussed there. Let the set of projects be U = {v, v, w, w}, and suppose that v A = v A , wA > wA and v  w  v  w: the manager can demand high and low-priority projects (v and v respectively), supply awards or schedule flexibility (w and w respectively), the worker’s costs from high and low-priority projects are the same, although the former are more valuable to the firm, and the worker prefers schedule flexibility to awards, although the latter are cheaper for the firm to provide. These projects are illustrated in Figure 2. We want to compare relationship dynamics in environments in which production possibilities are either growing or shrinking over time: the firm or its industry may be expanding or in decline, or the manager’s career within the firm, and hence her access to projects for her unit, may be promising or not. Our goal is to illustrate how the evolution of the relationship’s prospects, even if it has no impact on project prioritisation through cost-effectiveness, can have a powerful effect on its productive efficiency. Consider two project processes, ug and ud , where ug will represent growing production opportunities and ud declining ones. For simplicity, assume that demand opportunities arrive in a fixed sequence in both ug and ud : there exits {vt }t≥1,t odd such that (a) P(ugt = vt ) = P(udt = vt ) = 1 for all t odd, and (b) P(ugt = v) = P(udt = v) = 0 for all t even and all v ∈ {v, v}. While the flow of projects benefiting the firm is deterministic, the flow of projects used to compensate the agent is uncertain and depends on the firm’s growth prospects. Specifically, for all t even, 14

assume that (a) the availability of supply opportunities depends only on calendar time, so that for each project history ut−1 , P(ugt |ut−1 ) and P(udt |ut−1 ) depend on ut−1 only through t, that (b) P(ugt+2 = w) > P(ugt = w) for all w ∈ {w, w}, and that (c) P(udt+2 = w) < P(udt = w) and furthermore limt→∞ P(udt = w) = 0 for all w ∈ {w, w}. Suppose that the firm either starts out big and then declines or starts out small and becomes large, but that the total (expected discounted) quantity of supply opportunities are the same P P t−1 t−1 over its lifetime: ∞ P(ugt = w) = ∞ P(udt = w) for all w ∈ {w, w}. It follows that t=1 δ t=1 δ e the ex ante Pareto-efficient contracts associated to ug and ud are identical: κge = κd = κe . Suppose further that κe is such that W e = w, so that the principal always demands both v P t−1 and v and only supplies w. Because, for any t0 > 1 odd, we have that ∞ P(ugt = w) > t=t0 δ P∞ t−1 P(ugt = w), it follows that the optimal contract under process ug is efficient, i.e., that t=1 δ ∗ κg∗ = κe . However, it is also the case that κd 6= κe : the same total prospects for compensating the worker, but scaled in time differently, lead to severe inefficiencies. First, in a growing firm, the manager always demands both high and low-priority projects, but when the firm is declining, because limt→∞ P(udt ∈ {w, w}) = 0, the manager must eventually stop demanding all projects. Second, in a growing firm, the manager can compensate the worker throughout the relationship using awards only, but when the firm is declining the principal must eventually supply both awards and schedule flexibility whenever these are available. When the firm’s future prospects have become sufficiently dim, the manager starts committing to schedule flexibility to motivate the agent but, because w  v, she simultaneously stops demanding low-priority projects.

5

Markov Project Processes

Some important questions about the dynamics of optimal contracts cannot be addressed systematically without restricting attention to special cases of our general model. In particular, which histories of demands lead the principal to supply projects more generously, and what factors make some demands more expensive to the principal than others? In this section we assume that the project process u is Markov. This allows us to sharpen our results considerably, and in fact we explicitly construct optimal contracts in this case. Proposition 3. Suppose that the project process u is Markov, and fix any optimal contract κ∗ . For all demands v ∈ D, there exists a threshold supply project W v ∈ S such that, given any history ut with ut = v, ∗ , W v }. Wt∗ = max{Wt−1 

15

If the project process is Markov, then the optimal threshold project following any history can be characterised through a simple recursive rule which pins down its dynamics as a function of past demands. Specifically, each demand v ∈ D has an associated fixed threshold supply project W v . The supply threshold W v sets a baseline level of generosity towards the agent following any history (ut−1 , v) in which the principal demands v, but whether that threshold is implemented ∗ ∗ depends on the inherited threshold Wt−1 : if W v is more generous than Wt−1 , then Wt∗ is updated ∗ ∗ to W v , while if Wt−1 is more generous than W v , then Wt∗ remains fixed at Wt−1 . If projects v v v, v ∈ D are such that W is less cost-effective than W , then project v is unambiguously more expensive to demand for the principal than project v: in return for v, the principal must commit supply more, and demand less, from the agent in the future, and because optimal contracts never become less generous towards the agent the effects of having demanded v as opposed to v are persistent. This uncovers the relevant stationarity property of optimal contracts with Markov project processes: while an optimal contract will not specify similar production decisions following all histories with the same current project opportunity, it will specify similar production decisions following histories with the same current project opportunities that have met the same most expensive demand, irrespective of the other properties of these histories.13 The recursive construction of optimal supply thresholds in Proposition 3 seems to suggest that optimal contracts with Markov projects only depend on backward-looking incentives through past demands. However, forward-looking incentives are key to the construction of the fixed thresholds W v . More precisely, the thresholds W v are derived as solutions to an inductive 1 sequence of reduced problems: W v , the most expensive threshold for the principal, is the most expensive solution, over all v ∈ D, to the problem of finding the optimal contract for the principal subject to (a) the initial project being v (i.e., u1 = v) and (b) a single individual rationality 2 constraint for the agent at time 1 (i.e., UA,1 = 0); then W v , the second most expensive threshold for the principal, is the most expensive solution, over all v ∈ D \ {v 1 }, to the problem of finding the optimal contract for the principal subject to (a) u1 = v, (b) UA,1 = 0 and (c) the fact 1 that the contract transitions to the more generous W v whenever v 1 arrives; and so on. Because the agent expects the optimal contract’s future transitions to more generous thresholds, the expensiveness of a current demand is linked to the expensiveness of the future demands. An important remark is that the expensiveness of demands and their cost-effectiveness are unrelated. In particular, all the reduced problems involved in the construction of the thresholds W v are solved by contracts with project selection decisions ordered by cost-effectiveness, for the same reasons as in Sections 3 and 4. Expensiveness, on the other hand, is a measure 13

This is the amnesia property (Kocherlakota, 1996): an optimal contract disposes of history dependence whenever its terms are revised. The property fails for general project processes with arbitrary history dependence.

16

of the stringency of the agent’s individual rationality constraint following a demand, and it is determined by the principal’s ability to reward the agent for acceding to it. Intuitively, this depends on two factors: the absolute cost to the agent associated with the demand for v (indexed by |vA |); and the value to the agent of future project opportunities conditional on having reached project v, which depends on the discount factor δ and the project process u. Indeed, it is easy to construct examples in which demands that impose a smaller stage cost on the agent are actually more expensive for the principal, because the production opportunities that follow these demands are unfavourable to the agent. However, if the project process is iid, then the value to the agent of future project opportunities is history-independent. In that case, the expensiveness of the principal’s demands is determined solely by their agent’s stage costs. Corollary 2. If the project process u is iid, then, given any projects v, v ∈ D, W v  W v only if |v A | > |v A |.

6

Conclusion

We recap the main results by discussing their relationship to two key assumptions: transfers cannot be used to support production and the principal can commit to production decisions. Transfers. While we have not explicitly allowed for monetary payments between the principal and the agent, models with transfers are special cases of our general model. Indeed, a transfer of k dollars to the agent can be represented by a supply project mS = (−k, k) ∈ S, while a transfer of k dollars to the principal can be represented by a demand project mD = (k, −k) ∈ D, where mS and mD are equally cost-effective. The flexibility of the project process allows for different specifications of transfer opportunities. On the one hand, if all non-monetary projects are followed by transitions to both mS and mD and k is large, then transfers are always available and essentially unrestricted in size. On the other hand, if mS arrives at fixed intervals, then the principal has infrequent but regular opportunities to pay a bonus to the agent. While our results apply to all models with transfers, they provide specific implications for the use of money in the dynamic relationships captured by our environment. First, the principal’s ability to use transfers to reward the agent does not crowd out supply through production: the principal will not start paying the agent until she has committed to supply projects that are more cost-effective than money in all their future occurrences. Furthermore, in an optimal contract the principal may even supply projects that are less cost-effective than money, if the availability of future transfer opportunities is sufficiently constrained. However, if k is large and transfer opportunities are frequent, then the principal would always use money instead of less cost-effective projects. Second, the direction of the flow of money between the principal and 17

the agent varies over the relationship’s lifetime: the principal demands transfers from the agent early in the relationship, and supplies transfers to the agent later in the relationship. No commitment for the principal. If the principal cannot commit to production decisions, then individual rationality constraints for the principal cap her supply of projects. Cost-effectiveness still drives project selection decisions, but with an important qualification: if the principal supplies a less cost-effective project, then she must also supply more cost-effective projects in all succeeding histories in which none of her individual rationality have been binding. This implies that some characterisation of the optimal contract in terms of threshold supply projects would still be possible without commitment by the principal, but that pinning down general properties of optimal contracts’ dynamics would be difficult. Recall that if both sides can commit to production decisions, then the threshold supply project is fixed over time, and if only the principal has commitment power, then the threshold becomes more favourable to the agent over time to incentivise demands. If the principal cannot commit either, then she must have incentives to supply projects, which would imply a threshold that becomes less favourable to the agent following some histories. Therefore, in contrast with our results, the optimal contract will typically not stabilise in the long-run. Absence of commitment power for the principal would generate an inefficiency closely related to the one we discussed in Section 4: the principal and the agent would be better off if past demands could incentivise the principal’s current supply of projects, but without commitment these can only be supported by future demands.

A

Appendix

Proof of Lemma 1. Suppose, towards a contradiction, that κ∗ is optimal and that, for some ˜ that is project history ut such that uP,t , uA,t > 0, we have that κ∗t < 1. Fix a contract κ ∗ t identical to κ except that κ ˜ t = 1 at u . It follows that κ ˜ is individually rational because κ∗ is ∗ individually rational. Furthermore, U˜P,t > UP,t , yielding the desired contradiction. The proof t for the case of u such that uP,t , uA,t < 0 is similar, and is omitted. As we note in the text following Proposition 2, there can be indeterminacy in the production decisions specified by optimal contracts at threshold supply projects. We tackle this issue by selecting those optimal contracts that can be represented by time thresholds. This yields a characterisation of optimal contracts which is more complete than, but implies, the characterisation through threshold supply projects in Proposition 2. Specifically, for all projects u ∈ D ∪ S, the optimal contract will assign a history-contingent time threshold T u : H → {1, 2, . . .} ∪ {∞} where, given any project history ut we will, for notational simplicity, denote T u (ut ) by Ttu , with the history ut understood. For any project v ∈ D and given any history ut , the threshold Ttv 18

will denote the time at which the principal plans to stop demanding project v, and for projects w ∈ S, the threshold Ttw will denote the time at which the principal plans to start supplying project w. We will let Ttut denote the time threshold for the project that is realised at t, i.e., ut . Proposition 4. There exists an optimal contract which, for any project history ut , takes the following cutoff form: for all v ∈ D, there exists a time threshold Ttv such that    1   κ∗t = Ttvt − t    0

if Ttvt ≥ t + 1, if t < Ttvt < t + 1,

(5)

if Ttvt ≤ t;

and for all w ∈ S, there exists a time threshold Ttw such that    1   κ∗t = t + 1 − Ttwt    0

if Ttwt ≤ t, if t < Ttwt < t + 1,

(6)

if Ttwt ≥ t + 1.

Given any projects v and w, the thresholds Ttv and Ttw have the following properties. P0 1. Ttv0 = Ttv and Ttw0 = Ttw for all t0 > t such that ts=t κ∗s Ius ∈D = 0. [time thresholds are constant in between demands] 2. Ttv and Ttw are non-increasing in t. [time thresholds can only become more generous towards the agent] ∗ w v = 0. > Ttw , then UA,t > Ttv or Tt−1 3. If Tt−1 [time thresholds are only adjusted following binding individual rationality constraints]

P 4. Ttw = ∞ if ts=1 κ∗s Ius ∈D = 0. [no project is supplied before the first project is demanded] 5. Fix projects v  v. If Ttv < t + 1, then Ttv ≤ t. [if a demand is passed over, then so are less cost-effective demands] 6. Fix projects w  w. If Ttw > t, then Ttw = ∞. [if a supply project is not exhausted, then no less cost-effective supplies are promised] w 7. Let W t−1 = min {w ∈ S : Tt−1 > t + 1}. If v  W t−1 , then Ttv > t. [demand any project more cost-effective than the most cost-effective non-committed supply]

19

w < t + 2}. If W t−1  v, then Ttv ≤ t. Let W t−1 = max {w ∈ S : Tt−1 [do not demand any project less cost-effective than least cost-effective committed supply] ∗ w To relate Propositions 2 and 4, fix any history ut and define Wt−1 = max {w : Tt−1 < ∞} if ∗ w well-defined, with Wt−1 = min S otherwise. By part 1 of Proposition 4, we have that Ttw = Tt−1 ∗ and (b) Ttwt ≤ t, and if ut ∈ S, so that, by part 7, (a) Ttwt = ∞, and hence κ∗t = 0, if wt  Wt−1 ∗  wt , as required by part (i) of Proposition 2. By part 2 of Proposition hence κ∗t = 1, if Wt−1 0 t−1 ∗ ∗ 4, it follows that if u is a subhistory of ut −1 , then either Wt∗0 −1 = Wt−1 or Wt∗0 −1  Wt−1 , as w ∗ required by part (iii) of Proposition 2. Because Tt−1 = ∞ for all w  Wt−1 , it follows that if ∗ v  min {w  Wt−1 } then v  W t−1 , so that the first claim of part (ii) of Proposition 2 follows ∗ w  w, it ≤ t for all Wt−1 by the first claim of part 7 of Proposition 4. Similarly, because Tt−1 ∗ follows that if max {Wt−1  w}  v then W t−1  v, so that the second claim of part (ii) of Proposition 2 follows by the second claim of part 7 of Proposition 4.

Proof of Proposition 4. We proceed in a number of steps. Claims for the Optimal Supply of Projects 0 00 Step 1. Fix and optimal contract κ∗ , project history ut , its superhistories ut and ut , and P 00 projects w  w. Suppose that (i) ut0 = w and (ii) ut00 = w and ts=t κ∗s Ius ∈D = 0.14 We show that if κ∗t0 < 1, then κ∗t00 = 0. 0

00

To see this suppose, towards a contradiction, that κ∗t0 < 1 at ut and that κ∗t00 > 0 at ut . Now 0 consider an alternative contract κ ˜ , identical to κ∗ except that (i) κ∗t0 < κ ˜ t0 ≤ 1 at ut , (ii) 00 0≤κ ˜ t00 < κ∗t00 at ut and (iii) 00 00 0 0 ∗ U˜A,t − UA,t ˜ t00 ]wA = 0. = δ t −t Pt (ut )[˜ κt0 − κ∗t0 ]wA − δ t −t Pt (ut )[κ∗t00 − κ

(7)

Such a contract always exists, and furthermore κ ˜ is individually rational for the agent. To see ∗ this, first note that, because U˜A,t = UA,t ≥ 0, we have that κ ˜ satisfies (IRA,r ) for all times ∗ ˜ r ≤ t. Second, because UA,t0 > UA,t0 ≥ 0, it follows that given any time r > t and history ur 00 ∗ that is not a subhistory of ut , we have that U˜A,r ≥ UA,r ≥ 0. Third, even though we have that Pt00 ∗ t t00 ∗ Throughout, s=t κs Ius ∈D = 0 denotes that, given history u and its superhistory u , κs = 0 for any 00 history us with us ∈ D that is superhistory of ut and a subhistory of ut . 14

20

∗ U˜A,t00 < UA,t ˜ t00 ≥ 0 it also follows that 00 , because κ ∗ U˜A,t00 ≥ δEt00 UA,t 00 +1

≥ 0. 00

Finally, given time r > t and history ur which is a subhistory of ut , the fact that 0 implies that

Pt00

s=t

κ∗s Ius ∈D =

00 00 U˜A,r ≥ δ t −r Pr (ut )U˜A,t00

≥ 0. It remains only to note that, by (7), we have 00 00 0 0 ∗ ˜ t00 ]|wP | U˜P,t − UP,t = −δ t −t Pt (ut )[˜ κt0 − κ∗t0 ]|wP | + δ t −t Pt (ut )[κ∗t00 − κ   |wP |/w 00 00 A = δ t −t Pt (ut )[κ∗t00 − κ ˜ t00 ]|wP | 1 − |w | P /w A

> 0, where the inequality follows because w  w, contradicting the optimality of κ∗ . Step 2. Step 1 implies that to any optimal contract κ∗ corresponds a history-dependent threshold project mapping W : H → S such that, for all times t and histories ut , if ut ∈ S, then κ∗t =

 1

if Wt  ut ,

0

if ut  Wt ,

where for simplicity we denote W (ut ) by Wt , with the project history understood. Furthermore, Wt is non-decreasing (with respect to ), and is such that, given any history ut and its P0 0 superhistory ut , Wt0 = Wt if ts=t+1 κ∗s Ius ∈D = 0. The threshold is given by  Xt0 W (u ) = max w ∈ S : Pt (κ∗t0 > 0, ut0 = w, t



s=t+1

if this is well-defined, and by Wt = min S, 

21

κ∗s Ius ∈D

 = 0) > 0 ,

otherwise.15 Step 3. Step 2 does not determine the optimal contracts at times t when ut = Wt . We now show that, without loss of generality for optimal payoffs, we can restrict attention to contracts 0 with the property that, given any history ut and its superhistory ut at which ut0 = Wt and Pt0 ∗ 0 ∗ ˆ ˆ s=t+1 κs Ius ∈D = 0, there exists time T such that κt0 = 1 if and only if t ≥ T . More precisely, fix an optimal contract κ∗ and history ut , and consider an alternative contract κ ˆ , identical to P 0 0 t κ∗ except that, at all superhistories ut of ut with s=t+1 κ∗s Ius ∈D = 0 and ut0 = Wt ,    1   κ ˆ t0 = t0 + 1 − Tˆ    0

if Tˆ ≤ t0 , if t < Tˆ < t + 1, if Tˆ ≥ t0 + 1.

∗ ∗ Note that UˆA,t ≥ UA,t if Tˆ = t. Also, limTˆ→∞ UˆA,t ≤ UA,t . By continuity of UˆA,t in Tˆ, there ∗ . Also, note that exists some T˜ ≥ t such that U˜A,t = UA,t

" U˜P,t −

∗ UP,t

= |WP,t | E

# X

δ

t0 −t

[κ∗t0

−κ ˜ t0 ]

t0 ≥t

=

i |WP,t | h ∗ UA,t − U˜A,t WA,t

= 0. ∗ ≥ 0 either if To verify that κ ˜ is individually rational for the agent, first note that U˜A,r = UA,r P r (i) r ≤ t or if (ii) r > t and history ur is not a superhistory of ut with s=t+1 κ∗s Ius ∈D = 0. P Second, given any superhistory ur of ut with rs=t+1 κ∗s Ius ∈D = 0 and r > Tˆ, then, because ∗ κ ˜ s uA,s ≥ κ∗s uA,s for all s ≥ r, it follows that U˜A,r ≥ UA,r ≥ 0. Third, given any superhistory ur P of ut with rs=t+1 κ∗s Ius ∈D = 0 and r ≤ Tˆ, then, by the previous point and because κ ˜ r uA,r ≥ 0, it follows recursively that

U˜A,r ≥ δEr U˜A,r+1 ≥ 0. The last point is to establish that the procedure above, which modifies κ∗ at a single history in a payoff-invariant way, can be extended simultaneously to all histories. We do this in Step 7 Throughout, Pt (κ∗t0 > 0, ut0 = w,

Pt0

κ∗s Ius ∈D = 0) > 0 denotes that, given history ut , the set of its Pt0 superhistories ut such that κ∗t0 > 0, ut0 = w and s=t+1 κ∗s Ius ∈D = 0 has positive probability. 15

s=t+1

0

22

below. Therefore, this step defines a history-dependent time threshold TtWt for each history ut . P0 0 By construction, for any history ut and its superhistory ut with ut0 = Wt and ts=t+1 κ∗s Ius ∈D = 0 W (and hence Wt0 = Wt ), we have that TtWt = Tt0 t0 . Step 4. Step 3 defines a history-dependent time threshold TtWt for each history ut . We now show that, without loss of generality for optimal payoffs, we can restrict attention to contracts 0 with the property that, given any history ut and any superhistory ut such that Wt = Wt0 , we P0 W have that TtWt ≥ Tt0 t0 . By Step 3, given ut , if Pt (κ∗t0 > 0, ut0 = Wt , ts=t+1 κ∗s Ius ∈D = 0) = 0, then TtWt = ∞, and the claim is true. Therefore, in what follows we assume that, given ut , P0 Pt (κ∗t0 > 0, ut0 = Wt , ts=t+1 κ∗s Ius ∈D = 0) > 0. W Fix time t and history ut , and let T t = sup{Tt0 t0 : t0 ≥ t, ut0 = Wt = Wt0 }. It is possible, without loss of generality for optimal payoffs, to assume that T t < ∞. To see this, consider 0 an alternative contract κ ˆ , identical to κ∗ except that, given any superhistory ut of ut with P0 P0 W W W Wt = Wt0 , (i) Tˆt0 t0 = Tˆ if ts=t+1 κ∗s Ius ∈D = 0, and (ii) Tˆt0 t0 = min{Tt0 t0 , Tˆ} if ts=t+1 κ∗s Ius ∈D > ∗ ∗ because Pt (κ∗t0 > 0, ut0 = if Tˆ = t, and limTˆ→∞ UˆA,t < UA,t 0. We have that UˆA,t ≥ UA,t Pt0 ∗ , as well Wt , s=t+1 κ∗s Ius ∈D = 0) > 0. By continuity, there exists T˜ < ∞ such that U˜A,t = UA,t ∗ ˜ ˜ is individually rational for the agent, first note that as UP,t = UP,t . To verify that contract κ ∗ U˜A,r = UA,r ≥ 0 unless history ur is a superhistory of ut with Wr = Wt . Second, by construction ∗ ≥ 0 for any superhistory ur of ut with r ≥ T˜ and Wr = Wt as well as for of κ ˜ , U˜A,r ≥ UA,r P any superhistory ur of ut with t < r < T˜, Wr = Wt and rs=t+1 κ∗s Ius ∈D > 0. Third, given any P superhistory ur of ut with t < r < T˜, Wr = Wt and rs=t+1 κ∗s Ius ∈D = 0, then, by the previous points and because κ ˜ r uA,r ≥ 0, it follows recursively that U˜A,r ≥ δEr U˜A,r+1 ≥ 0. Now consider an alternative contract κα , identical to κ∗ except that, given any superhistory P0 0 α,W α,W ut of ut with Wt = Wt0 , (i) Tt0 t0 = (1 − α)T t + αt if ts=t+1 κ∗s Ius ∈D = 0, and (ii) Tt0 t0 = P0 W (1 − α)Tt0 t0 + αt if ts=t+1 κ∗s Ius ∈D > 0. This contract is well-defined because T t < ∞. Notice α=0 ∗ α=1 ∗ that UA,t ≤ UA,t and that UA,t ≥ UA,t . By continuity, there exists α ˜ ∈ [0, 1] such that W α ˜ ∗ α ˜ ∗ UA,t = UA,t , as well as UP,t = UP,t . Note that, by construction, because T t ≥ Tt0 t0 , κα˜ is such α,W ˜ ˜ t that Ttα,W ≥ Tt0 t0 whenever t0 > t and Wt0 = Wt . The proof that κα˜ is individually rational for the agent is almost identical to that of the previous paragraph for the contract κ ˜ , and is omitted. Step 5. We use this step to prove the one that follows. Fix an optimal contract κ∗ , project 0 00 history ut−1 , its superhistories ut and ut , a superhistory ut of ut , and projects w  w. Suppose

23

Pt00

that (i) ut0 = w, and (ii) ut00 = w and

s=t+1

∗ κ∗s Ius ∈D = 0. We show that if UA,t > 0 at ut , then

if κ∗t0 < 1, then κ∗t00 = 0. 0

00

To see this suppose, towards a contradiction, that κ∗t0 < 1 at ut and that κ∗t00 > 0 at ut . Now 0 consider an alternative contract κ ˜ , identical to κ∗ except that (i) κ∗t0 < κ ˜ t0 ≤ 1 at ut , (ii) 00 0≤κ ˜ t00 < κ∗t00 at ut , (iii) U˜A,t ≥ 0 and (iv) 00 00 0 0 ∗ ˜ t00 ]wA = 0. (8) U˜A,t−1 − UA,t−1 = δ t −(t−1) Pt−1 (ut )[˜ κt0 − κ∗t0 ]wA − δ t −(t−1) Pt−1 (ut )[κ∗t00 − κ

Such a contract always exists, and furthermore κ ˜ is individually rational for the agent. To see ∗ ˜ this, first note that, because UA,t−1 = UA,t−1 ≥ 0, we have that κ ˜ satisfies (IRA,r ) for all times r ≤ t − 1. Second, U˜A,t ≥ 0 so that κ ˜ satisfies (IRA,t ). Third, κ ˜ satisfies (IRA,r ) for all times r > t. This follows by an argument similar to the one in Step 1 and is omitted. Finally, an argument as in Step 1 shows that (8) and the fact that w  w imply that ∗ ˜ UP,t−1 − UP,t−1 > 0, yielding the desired contradiction. Step 6. We show that, without loss of generality for optimal payoffs, we can restrict attention to contracts κ∗ such that, given any time t, if either (i) Wt  Wt−1 or (ii) Wt = Wt−1 and W ∗ Tt−1t−1 > TtWt , then UA,t = 0. To see part (i) of this claim, suppose, towards a contradiction, ∗ that there exist project history ut−1 and its superhistory ut such that Wt  Wt−1 and UA,t > 0. ∗ Because Wt  Wt−1 , Step 2 implies that ut ∈ D, κt > 0 and  Pt

κ∗t00

> 0, ut00 = Wt ,

Xt00 s=t+1

κ∗s Ius ∈D

 =0 >0

(9)

P 00 00 so that there exists superhistory ut of ut with ut00 = Wt , ts=t+1 κ∗s Ius ∈D = 0 and κ∗t00 > 0. We now make two claims. First, we claim that project w ∈ S such that Wt  w  Wt−1 does not exist. Suppose it does, then, by Step 2,   Xt0 ∗ ∗ κs Ius ∈D = 0 = 0 Pt−1 κt0 > 0, ut0 = w, s=t

0

so that there exists a superhistory ut of ut−1 with ut0 = w and κ∗t0 = 0, which, by Step 5 and (9), is a contradiction because Wt  w. Second, we claim that   Xt0 ∗ ∗ Pt−1 κt0 < 1, ut0 = Wt−1 , κs Ius ∈D = 0 = 0. s=t

0

If not then there exists superhistory ut of ut−1 with ut0 = Wt−1 and κ∗t−1 < 1, which, by Step 5 24

and (9), is a contradiction because Wt  Wt−1 . α,β,Wt0 Now consider the alternative contract κα,β , identical to κ∗ except that (i) Tt0 =β≥t P 0 0 t for all superhistories ut of ut with Wt0 = Wt and s=t+1 κ∗s Ius ∈D = 0 and (ii) Wtα,β = Wt 00 Pt00 ∗ α,β,Wt t00 t−1 and Tt00 = α ≥ t for all superhistories u of u with s=t κs Ius ∈D = 0. We have that t,t α,β ∗ ∗ UA,r ≥ UA,r and limα,β→∞ UA,r ≤ UA,r for r = t − 1, t. By continuity, there exist β˜ ≤ α ˜<∞ ˜ ˜ α, ˜β α, ˜β α, ˜ β˜ ∗ ∗ such that UA,t−1 = UA,t−1 and UP,t−1 = UP,t−1 , and either (a) UA,t = 0 and β˜ ≤ α ˜ or (b) ˜ ˜ α, ˜ β˜ α, ˜ β α, ˜ β U > 0 and β˜ = α ˜ , which implies W = Wt by the two claims above. t−1

A,t

˜

˜β Contract κα, is clearly individually rational for the agent at all times r ≤ t. The proof that ˜ ˜β contract κα, is individually rational for the agent at all times r > t is similar to those of Steps 3 and 4, and is omitted. Finally, the proof of part (ii) of the claim is similar, and is omitted. Step 7. The payoff-equivalent modifications operated on optimal contract κ∗ described in Steps 3-6 were constructed history by history. Note that any contract can be identified with a point in [0, 1]∞ , a compact set in the product topology. Therefore, given an optimal contract κ∗,1 , we can construct a sequence {κ∗,n }n≥1 in [0, 1]∞ such that (i) for each n, κ∗,n+1 is obtained from κ∗,n by some operation from Steps 3-6 at some history and (ii) given any time t, there exists 0 ∗,N N such that, for all n ≥ N , κ∗,n for all histories ut with t0 ≤ t. This sequence must t0 = κt0 then have a subsequence converging to κ∗ , some optimal contract satisfying all the properties of Steps 3-6. Step 8. Given any time t and associated threshold project Wt as defined in Step 2, we have defined in Steps 3-7 a time threshold TtWt that respects the conditions of Proposition 4. Now given any w  Wt , define Ttw = ∞, and given any Wt  w, define

 0 min{T W t0 : ut is a subhistory of ut and w = Wt0 } t0 w Tt = min{t0 : ut0 is a subhistory of ut and W 0  w} t

if this is well-defined, otherwise.

Note that because, by Step 2, W is non-decreasing (with respect to ), and because, by Steps 4 and 6, TtWt is non-increasing in t, our construction ensures that, for each w ∈ S, the threshold Ttw is non-increasing in t. Step 9. Let w = min S. We show that given an optimal contract κ∗ , we have that Ttw = ∞ if Pt Pt ∗ t ∗ s=1 κs Ius ∈D = 0. To see this, suppose that there exists a history u with s=1 κs Ius ∈D = 0 and, ˜ , identical to κ∗ except towards a contradiction, Ttw < ∞. Consider an alternative contract κ P 0 0 that κ ˜ t0 = 0 at all histories ut with ut0 ∈ S and ts=1 κ∗s Ius ∈D = 0. To see that κ ˜ is individually Pr ∗ rational for the agent, first note that if s=1 κ∗s Ius ∈D > 0 for some r, then U˜A,r = UA,r ≥ 0. Pr ∗ Second, if s=1 κs Ius ∈D = 0 for some r, then, because κ ˜ r uA,r ≥ 0, it follows recursively by the

25

previous point that we have ∗ U˜A,r ≥ δEr UA,r+1

≥ 0. ∗ Finally, we have that U˜P,1 > UP,1 , yielding the desired contradiction. Claims for the Optimal Demand for Projects 0 Step A. Fix optimal contract κ∗ , project history ut , its superhistory ut , and projects v  v. Suppose that (i) ut = v and (ii) ut0 = v. We show that

if κ∗t < 1, then κ∗t0 = 0. 0

To see this suppose, towards a contradiction, that κ∗t < 1 at ut and that κ∗t0 > 0 at ut . Now ˜ t ≤ 1 at ut , (ii) consider an alternative contract κ ˜ , identical to κ∗ except that (i) κ∗t < κ 0 0≤κ ˜ t0 < κ∗t0 at ut , and (iii) 0 0 ∗ U˜A,t − UA,t = − [˜ κt − κ∗t ] |v A | + δ t −t Pt (ut ) [κ∗t0 − κ ˜ t0 ] |v A | = 0.

(10)

Note that contract κ ˜ satisfies (IRA,r ) at all times r ≤ t. To show that κ ˜ satisfies (IRA,r ) at all ∗ ˜ times r > t, we proceed recursively. First note that we have that UA,r = UA,r ≥ 0 whenever (i) 0 ur is not a superhistory of ut or (ii) r > t0 . Second, at history ut , we have that κ ˜ t0 uA,t0 > κ∗t0 uA,t0 , ∗ ≥ 0. Third, given any superhistory ur of ut with so that, by the previous point, U˜A,r > UA,r t < r < t0 , the previous points ensure that U˜A,r ≥ 0. Finally, an argument as in Step 1 shows ∗ > 0, yielding the desired contradiction. that (10) and the fact that v  v imply that U˜P,t − UP,t 0 Step B. Step A does not restrict optimal contracts at history ut and its superhistory ut if ut = ut0 ∈ D. We now show that, without loss of generality for optimal payoffs, we can restrict attention to contracts with the property that, for such histories, if κ∗t0 > 0, then κ∗t = 1. To see this, fix optimal contract κ∗ , history ut and project v, and suppose that ut = v and κ∗t < 1. Now consider an alternative contract κ ˜ , identical to κ∗ except that

κ ˜t =

 κ∗ + E P t

1

t

s−t ∗ κs Ius =v s≥t+1 δ



if κ∗t + Et

otherwise,

26

 s−t ∗ δ κ I ≤ 1, u =v s s s≥t+1

P

0

and that, for any superhistory ut of ut with ut0 = v,   0  0 κ ˜t =   1−

if κ∗t + Et 1−κ∗t P Et [ s≥t+1 δ s−t κ∗s Ius =v ]



κ∗t0

P

s≥t+1

 δ s−t κ∗s Ius =v ≤ 1,

otherwise.

Note that such a contract always exists, and that, by construction, "

"

∗ U˜A,t − UA,t = |vA | − [˜ κt − κ∗t ] + Et

## X

δ s−t Ius =v [κ∗s − κ ˜s]

s≥t+1

=0 ∗ = U˜P,t − UP,t . 0

Furthermore, for any superhistory ut of ut with ut0 = v, we have that either (i) κ ˜ t = 1 and κ ˜ t0 ≥ 0, or (ii) κ ˜ t < 1 and κ ˜ t0 = 0. Note that contract κ ˜ satisfies (IRA,r ) for all r ≤ t. To see that κ ˜ satisfies (IRA,r ) at all r > t, note that because κ ˜ r ≤ κ∗r for all superhistories ur of ut , it ∗ ≥ 0. follows that κ ˜ r uA,r ≥ κ∗r uA,r for all r > t, and hence V˜A,r ≥ VA,r ∗ Step C. The procedure from Step B, which modifies contract κ at a single history in a payoffinvariant way, can be extended simultaneously to all histories as in Step 7. Step D. Given an optimal contract κ∗ along with any v ∈ D and any history ut , define 0

v

t = sup{t0 : ut is a subhistory or superhistory of ut , vt0 = v and κ∗t0 > 0} v

if this is well-defined and t = ∞ otherwise, as well as Ttv

=

 tv + κ∗v

if t < ∞,

∞

otherwise.

t

v

By construction, the resulting time thresholds {Ttv }v∈D are non-increasing. Furthermore, by the results of Steps A-C, it follows that, for all t,    1 if Ttvt ≥ t + 1,   κ∗t = Ttvt − t if t < Ttvt < t + 1,    0 if Ttvt ≤ t. Step E. Fix optimal contract κ∗ and project history ut such that vt  W t−1 . We show that κ∗t > 0. Suppose, towards a contradiction, that κ∗t = 0. Then by Part 1 of Proposition 4 (which 27

W

W

we have established), at superhistory ut+1 of ut with ut+1 = W t−1 , Tt+1t−1 = Tt−1t−1 > t + 1 and ˜ , identical to κ∗ except that (i) κ ˜ t > 0, hence κ∗t+1 < 1. Now consider an alternative contract κ ∗ ∗ ˜ t+1 ≤ 1 and (iii) U˜A,t = UA,t . Such a contract always exists. To see that κ ˜ is (ii) κt+1 < κ individually rational for the agent, first note that it is individually rational for the agent at all histories except at ut+1 . Second, for ut+1 , because κ ˜ t+1 uA,t+1 ≥ 0, it follows recursively by the previous point that we have ∗ U˜A,t+1 ≥ δEt+1 UA,t+2

≥ 0. By (iii), we have that 
 ∗ U˜A,t − UA,t = −˜ κt |vA,t | + δW A,t−1 κ ˜ t+1 − κ∗t+1 Pt ut+1 = W t−1

(11)

= 0. But then, it follows that  
∗ U˜P,t − UP,t =κ ˜ t vP,t − δ W P,t−1 κ ˜ t+1 − κ∗t+1 Pt ut+1 = W t−1 " # W vP,t P,t−1 =κ ˜ t |vA,t | − |vA,t | W A,t−1 > 0, where the second equality follows from substituting (11) and the inequality follow because vt  W t−1 , contradicting the optimality of κ∗ . Step F. Fix optimal contract κ∗ and project history ut such that W t−1  vt . We show that κ∗t = 0. Suppose, towards a contradiction, that κ∗t > 0. Then by Part 2 of Proposition 4 (which W W we have established), at superhistory ut+1 of ut with ut+1 = W t−1 , Tt+1t−1 ≤ Tt−1t−1 < t + 2 ˜ , identical to κ∗ except that (i) and hence κ∗t+1 > 0. Now consider an alternative contract κ ∗ 0≤κ ˜ t < κ∗t , (ii) 0 ≤ κ ˜ t+1 < κ∗t+1 and (iii) U˜A,t = UA,t . Such a contract always exists and that κ ˜ is individually rational for the agent follows by an argument as in Step E. By (iii), we have that  
∗ U˜A,t − UA,t = [κ∗t − κ ˜ t ] |vA,t | − δW A,t−1 κ∗t+1 − κ ˜ t+1 Pt ut+1 = W t−1 = 0.

28

(12)

But then, it follows that  
∗ ˜ t+1 Pt ut+1 = W t−1 U˜P,t − UP,t = − [κ∗t − κ ˜ t ] vP,t + δ W P,t−1 κ∗t+1 − κ # " W P,t−1 v P,t − = [κ∗t − κ ˜ t ] |vA,t | W A,t−1 |vA,t | > 0, where the second equality follows from substituting (12) and the inequality follow because W t−1  vt , contradicting the optimality of κ∗ . Our characterisation of optimal contracts with Markov project process in Proposition 3 ∗ shows how to define the cutoff supply project Wt−1 from Proposition 2 through a recursive rule involving fixed threshold {W v }v∈D associated to all demand projects. Our proof of this result will follow from the next result, which shows how to construct the optimal time thresholds {Ttv }v∈D and {Ttw }w∈S from Proposition 4 through a recursive rule involving fixed time thres0 0 holds {{τ v v }v∈D , {τ v w }w∈S }v0 ∈D . As was the case for Proposition 4 relative to Proposition 2, Proposition 5 implies Proposition 3, but provides a more detailed characterisation of optimal contracts. Proposition 5. Suppose that the project process u is Markov. For all v, v 0 ∈ D and all w ∈ S, 0 0 there exist τ v v , τ v w ≥ 0 that recursively define optimal time thresholds Ttv and Ttw which satisfy the conditions of Proposition 4. 1. T0v = T0w = ∞ for all v ∈ D and w ∈ S. 2. Given any t ≥ 1, Ttv

=

 t + τ v t v

v if t + τ vt v < Tt−1 ,

T v

otherwise,

t−1

Ttw

=

 t + τ vt w

w if t + τ vt w < Tt−1 ,

T w

otherwise,

t−1

for all v ∈ D, and

for all w ∈ S.

Furthermore, given any projects v, v ∈ D, if either τ vv ≤ τ vv for some v ∈ D or τ vw ≤ τ vw for some w ∈ S,

29

then both τ vv ≤ τ vv for all v ∈ D and τ vw ≤ τ vw for all w ∈ S. Given some demand project v and any project u, suppose that the principal can demand v following some history ut . Then τ vt u indicates either (i) the amount of time for which the principal will demand u ∈ D in superhistories of ut or (ii) the amount of time after which the principal will supply u ∈ S in superhistories of ut . To relate Propositions 3 and 5, fix 0 demand project v ∈ D and let W v = max {w : τ vw < ∞} if well-defined, with W v = min S otherwise. The ranking of demand projects by their expensiveness to the principal (the final v v claim of Proposition 5) implies that if τ vW ≤ τ vW , then either W v  W v or W v = W v . The v v latter case can hold even if τ vW < τ vW , which is why Proposition 5 gives a finer description of optimal contracts than Proposition 3. Proof of Proposition 5. We proceed in a number of steps. Step 1. Fix project v 0 ∈ D and suppose that u1 = v 0 . We define the reduced problem max UP,1 subject to UA,1 ≥ 0.

(13)

κ∈K

∗ = 0 at any solution to (13). Second, First, note that a standard argument establishes that UA,1 note that problem (13) only requires individual rationality at t = 1. Therefore, if the solution to (13) also satisfies (IRA,t ) at all times t > 1, then it must be part of an optimal contract conditional on u1 = v 0 . Step 2. We show that there exists a solution κ∗ to (13) of the following threshold type: for each v ∈ D, there exists T v ≥ 0 such that, given any history ut with ut = v,

   1   ∗ κt = T v − t    0

if T v ≥ t + 1, if t < T v < t + 1, if T v ≤ t,

and for each w ∈ S, there exists T w ≥ 0 such that, given any history ut with ut = w,    1   ∗ κt = t + 1 − T w    0

if T w ≤ t, if t < T w < t + 1, if T w ≥ t + 1.

The critical difference with the corresponding expressions with a general process u from Propo30

sition 4 is that the time thresholds ({T v }v∈D , {T w }w∈S ) are fixed and independent of histories. The proof of this claim follows from arguments closely mirroring those of Steps 3 and C of Proposition 4, and is omitted. In fact, these arguments are simplified in this case because the only individual rationality constraint for the agent in problem (13) is for the initial history. Finally, we normalise these time thresholds so that (i) T v = 0 if and only if κ∗t = 0 for all t ≥ 1 with ut = v, that is, if and only if the principal never demands project v under κ∗ , and that (ii) T w = 0 if and only if κ∗t = 1 for all t ≥ 1 with ut = w, that is, if the principal always supplies project w under κ∗ . Step 3. We show that there exists a solution to (13) with the following properties: there exist v ∗ ∈ D and w∗ ∈ S such that 1. T v = 0 if v ∗  v and T v = ∞ if v  v ∗ . 2. T w = 0 if w∗  w and T w = ∞ if w  w∗ . 3. Given any v ∈ D, if T v < ∞, then T w = 0 for all v  w. Also, if T v > 0 then T w = ∞ for all w  v. The threshold projects are defined as  min {v ∈ D : T v > 0} if well-defined,  v∗ = max D if T v = 0 for all v ∈ D,  and  max {w ∈ S : T w < ∞} if well-defined,  w∗ = min S if T w = ∞ for all w ∈ S.  In words, v ∗ is the worst project among those which the principal ever demands, and w∗ is worst project among those the principal ever supplies (if applicable). Note that Item 3 implies that if ∗ ∗ ∗ ∗ T v < ∞, then T w = 0, and that if T w > 0, then T v = ∞. The proof of this claim follows from arguments closely mirroring those of Steps 1 and A of Proposition 4, and is omitted. Again, these arguments are simplified in this case because the only individual rationality constraint for the agent in problem (13) is for the initial history. Step 4. Given v, v ∈ D, consider the associated solutions κ∗ and κ∗ to the problem (13) with u1 = v and u1 = v, respectively. We show that if either (i) v ∗  v ∗ or (ii) v ∗ = v ∗ and T

v∗

∗

< Tv , 31

then either (i) w∗  w∗ or (ii) w∗ = w∗ and T

w∗

∗

≤ Tw . v∗

∗

To see this, suppose that either (i) v ∗  v ∗ or (ii) v ∗ = v ∗ and T < T v . Note that, by v Item 1 in Step 3, we have in both cases (i) and (ii) that T ≤ T v for all v ∈ D, with at least one inequality strict, so that, in words, the contract κ∗ is strictly more generous in terms of what it demands from the agent than κ∗ . Now suppose, towards a contradiction, that either ∗ w∗ (i) w∗  w∗ or (ii) w∗ = w∗ and T w < T . Note that, by Item 2 in Step 3, we have that w T ≥ T w for all w ∈ S, with at least one inequality strict, so that, in words, the contract κ∗ is strictly more generous in terms of what it supplies to the agent than κ∗ . First, let v˜ be such v˜ v˜ that T < T v˜, which by assumption must exist. Then, we have T < ∞ and T v˜ > 0. Second, w ˜ fix w˜ such that T > T w˜ , which by (our contradiction) assumption must exist. Then, we have w ˜ v˜ w ˜ T > 0 and T w˜ < ∞. Third, T < ∞ and T > 0, by Item 3 of Step 3, implies w˜  v˜. Fourth, ˜ yielding the desired contradiction. T v˜ > 0 and T w˜ < ∞, by Item 3 of Step 3, implies v˜  w, Step 5. The previous point allows us to rank the solutions to (13) for various v 0 ∈ D for which u1 = v 0 in terms of how expensive they are to the principal. Specifically, fix v, v ∈ D and consider the associated solutions κ∗ and κ∗ to the problem (13) with u1 = v and u1 = v, ∗ ∗ v∗ v∗ respectively. If either (i) v ∗  v ∗ , or (ii) v ∗ = v ∗ and T < T v , or (iii) v ∗ = v ∗ , T = T v and ∗ ∗ v∗ w∗ w∗  w∗ , or (iv) v ∗ = v ∗ , T = T v , w∗ = w∗ and T < T w , then we say that the contract κ∗ is more expensive for the principal than contract κ∗ . In words, Step 4 says that when these conditions are met, then κ∗ demands less of every project v ∈ D, and supplies more of every project w ∈ S, than κ∗ . Fix any project u such that u1 = u. and let U i,1 denote the payoff to i from contract κ∗ , and U i,1 denote the payoff to i from contract κ∗ . It follows that if κ∗ is more expensive for the principal than κ∗ , then we have that U A,1 ≥ U A,1 . An implication is that contract κ∗ must still satisfy (IRA,1 ) if u1 = v, but that contract κ∗ does not satisfy (IRA,1 ) if u1 = v. Step 6. We remove from the set D any project v for which the solution to problem (13) with u1 = v is the no-production contract following all histories ut with ut ∈ D ∪ S. For all histories in which one of these project is available, we set the production probabilities to 0 in the optimal contract we are constructing. Note that by the construction of problem (13), no individually rational contract delivers a higher payoff to the principal following any such history. Step 7. Let v 1 ∈ D be the project for which the solution κ1∗ to problem (13) with u1 = v 1 is the most expensive among all solutions to (13) with u1 = v 0 for some v 0 ∈ D. By Step 6, we can assume that κ1∗ is such that the associated time thresholds have T v > 0 for some v ∈ D (and correspondingly T w < ∞ for some w ∈ W). 32

First, we show that we cannot have v ∗  v 1 . In words, it must be that, conditional on u1 = v 1 , the principal makes a demand with positive probability at t = 1 under κ1∗ . To see this suppose, towards a contradiction, that v ∗  v 1 . Fix an initial history with u1 = v 1 , and define an alternative contract κ ˜ such that (i) κ ˜ 1 = 0 and κ ˜ t = 0 following any history ut such 0 that ut0 ∈ S for any subhistory ut of ut with t0 ≥ 2, and (ii) κ ˜ implements κ1∗ starting from t following all other histories. By the fact that u is a Markov process and by Step 5, we know that U˜A,t ≥ 0 following all histories listed in point (ii). In turn, because κ ˜ t uA,t = 0 for all histories listed in point (i), it follows that U˜A,t ≥ 0 for these histories. Therefore, U˜A,1 ≥ 0. Because 1∗ there exists some w ∈ S such that T w < ∞, we have that U˜1,P > U1,P , which contradicts the 1∗ optimality of κ in (13). Second, we show that following any history ut with t ≥ 2, contract κ1∗ satisfies (IRA,t ). To 1∗,t see this, let UA,t be the payoff to the agent if κ1∗ was implemented starting from t, and note that 1∗,t 1∗ ≥ UA,t UA,t

≥ 0, where the first inequality follows by the fact that u is a Markov process and because contract κ1∗ becomes more generous between times 1 and t, and the second inequality follows, again, by the fact that u is a Markov process and by Step 5. Finally, note that no individually rational contract delivers a higher payoff to the principal than κ1∗ at any history ut with ut = v 1 , which follows by the construction of problem (13). Step 8. Define the set of projects V 1 = {v 1 } with associated set of contracts K 1 = {κ1∗ }. Now, inductively, fix a set of projects V n−1 = {v 1 , . . . , v n−1 } and associated set of contract K n−1 = {κ1∗ , . . . , κn−1∗ }. Assume that (i) each κi∗ is individually rational following all histories, and that (ii) no individually rational contract delivers a higher payoff to the principal than κi∗ following any history ut with ut = v i . Further assume that (iii) the time thresholds associated to the contracts in K n−1 are such that T u,1 ≤ T u,2 ≤ . . . ≤ T u,n−1 for all u ∈ D ∪ S. Fix any project v 0 ∈ D \ V n−1 and suppose that u1 = v 0 . We define the reduced problem max UP,1 subject to UA,1 ≥ 0, κ∈K

UA,t ≥ 0 at each t > 1 at which ut ∈ V n−1 ,

(14)

κ = κi∗ at each t > 1 with ut = v i ∈ V n−1 and UA,t = 0. This problem corresponds closely to the problem (13): the goal is to find an optimal contract

33

while ignoring all individual rationality constraints other than (i) the constraint at time t = 1 but also (ii) all constraints associated with the first arrival of an opportunity to demand project v i ∈ V n−1 . In the latter case, the problem (14) prescribes that the contract κi∗ be adopted at this history if the agent’s individual rationality constraint binds. Step 9. As in Step 2 for problem (13), we show that there exists a solution to problem (14) with contract κ∗ of the following threshold type: for each v ∈ D \ V n−1 , there exists T v ≥ 0 such that, given any history ut with (i) ut = v and (ii) UA,t0 > 0 for all 1 ≤ t0 ≤ t for which ut0 ∈ V n−1 ,    1   ∗ κt = T v − t    0

if T v ≥ t + 1, if t < T v < t + 1, if T v ≤ t,

and for each w ∈ S, there exists T w ≥ 0 such that, given any history ut with (i) ut = w and (ii) UA,t0 > 0 for all 1 ≤ t0 ≤ t for which ut0 ∈ V n−1 ,

κ∗t =

   1  

t + 1 − Tw    0

if T w ≤ t, if t < T w < t + 1, if T w ≥ t + 1.

In words, the thresholds above are valid until the contract transitions to κi∗ for some i = 1, . . . , n − 1. As for Step 2, the proof of this claim follows from arguments closely mirroring those of Steps 3 and C of Proposition 4, and is omitted. Also as in Step 2, we normalise these time thresholds so that (i) T v = 0 if and only if κ∗t = 0 for all t ≥ 1 with ut = v, and that (ii) T w = 0 if and only if κt = 1 for all t ≥ 1 with ut = w. Step 10. Given simple adaptations of the arguments in Steps 3-5 for problem (13), it can be shown that solutions to (14) for projects v ∈ D \V n−1 can be ranked according to how expensive they are for the principal. Furthermore, each of these solutions is not the no-production contract, because each solution in K n−1 is not the no-production contract (by Step 6). Step 11. Let v n ∈ D \ V n−1 be the project for which the solution κn∗ to problem (14) with u1 = v n is the most expensive among all solutions to (14) with u1 = v 0 for some v 0 ∈ D \ V n−1 0 that also have T v > 0. First, if no such project exists, then for all projects v ∈ D \ V n−1 and all histories ut such that ut0 ∈ / V n−1 for all t0 ≤ t, we set the production probabilities to 0 in the optimal contract we are constructing. Furthermore, by construction of problem (14), no individually rational contract delivers a higher payoff to the principal following any such

34

history. Second, if instead project v n is well-defined, then simple adaptations of the arguments in Step 7 for problem (13) ensure that contract κn∗ is such that (i) it satisfies (IRA,t ) following any history ut and (ii) no individually rational contract delivers a higher payoff to the principal than κn∗ at any history ut with ut = v n . Step 12. The previous step concludes the inductive construction of the optimal contract. Note that for each contract in the collection K n , the associated thresholds ({T v }v∈D , {T w }w∈S ) define
0 0 the collection of thresholds {τ v v }v∈D , {τ v w }w∈S v0 ∈K n from the statement of the proposition. It only remains to be shown that these thresholds are ordered by their expensiveness for the u principal: suppose that v = v n and that v = v n−1 , then it follows that T ≤ T u for all u ∈ D ∪S. To see this, first note that, if we let {T u,n−1 }u∈D∪S be the thresholds associated to the solution to the version of problem (14) at stage n − 1 (given sets V n−2 and K n−2 ) with u1 = v, we u have that T ≤ T u,n−1 for all u ∈ D ∪ S. Second, our claim is established after we show that T u,n−1 ≤ T u for all u ∈ D ∪ S. To see this, note that given u1 = v, problem (14) differs from the version of this problem at stage n − 1 only through the additional constraint that κ = κ following all histories with ut = v and UA,t = 0. Therefore, if we let U z,n−1 denote the agent’s A,1 u,n−1 }u∈D∪S , it follows that payoff in (14) at stage z = n − 1, n with u1 = v to thresholds {T n,n−1 n−1,n−1 u U A,1 ≥ U A,1 = 0. Given this, it must be that thresholds {T }u∈D∪S are less expensive u,n−1 }u∈D∪S , that is, that T u,n−1 ≤ T u for all u ∈ D ∪ S. than thresholds {T Proof of Corollary 2. Suppose that the project process u is iid, fix some history ut along with a contract κ, consider the agent’s payoff UA,t = κt uA,t + δEt UA,t+1 , and note that Et UA,t+1 is independent of ut . It follows that the solution to problem (13) with u1 = v is more expensive for the principal than the solutions to (13) with u1 = v if and only if |v A | > |v A |. The same property holds for solutions to problem (14), which together generate the ranking of projects v ∈ D in terms of their expensiveness for the principal.

References Abreu, D., D. Pearce, and E. Stacchetti (1990). Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58 (5), 1041–1063. Bird, D. and A. Frug (2017). Dynamic nonmonetary incentives. mimeo. Board, S. (2011). Relational contracts and the value of loyalty. American Economic Review 101 (7), 3349–3367. 35

Dixit, A., G. M. Grossman, and F. Gul (2000). The dynamics of political compromise. Journal of Political Economy 108 (3), 531–568. Guo, Y. and J. H¨orner (2015). Dynamic mechanism without money. mimeo. Hauser, C. and H. A. Hopenhayn (2008). Trading favors: Optimal exchange and forgiveness. mimeo. Kocherlakota, N. R. (1996). Implications of efficient risk sharing without commitment. Review of Economic Studies 63 (4), 595–609. Lazear, E. P. (1981). Agency, earnings profiles, productivity, and hours restrictions. American Economic Review 71 (4), 606–620. Levin, J. (2003). Relational incentive contracts. American Economic Review 93 (3), 835–857. Lipnowski, E. and J. Ramos (2016). Repeated delegation. mimeo. Milkovich, G., J. Newman, and B. Gerhart (2013). Compensation (11 ed.). New York, NY: McGraw-Hill Education. Mirrlees, J. A. (1976). The optimal structure of incentives and authority within an organization. Bell Journal of Economics 7 (1), 105–131. M¨obius, M. M. (2001). Trading favors. mimeo. Ray, D. (2002). The time structure of self-enforcing agreements. Econometrica 70 (2), 547–582. Rogerson, W. P. (1985). Repeated moral hazard. Econometrica 53 (1), 69–76. Samuelson, L. and E. Stacchetti (2017). Even up: Maintaining relationships. Journal of Economic Theory 169, 170–217. Sannikov, Y. (2008). A continuous-time version of the principal-agent problem. Review of Economic Studies 75 (3), 957–984. Spear, S. E. and S. Srivastava (1987). On repeated moral hazard with discounting. Review of Economic Studies 54 (4), 599–617. Thomas, J. P. and T. Worrall (1988). Self-enforcing wage contracts. Review of Economic Studies 55 (4), 541–553. Thomas, J. P. and T. Worrall (1994). Foreign direct investment and the risk of expropriation. Review of Economic Studies 61 (1), 81–108. Thomas, J. P. and T. Worrall (2018). Dynamic relational contracts under complete information. Journal of Economic Theory 175, 624–651.

36