COPING WITH UNCERTAINTY: NORMATIVE APPROACHES, CURRENT PRACTICE May 22—24, 2017 Ecole Normale Supérieure 45 rue d’Ulm, 75005 Paris
PROGRAM MONDAY, MAY 22 09:15 — 09:45
coffee
09:45 — 10:00
Welcome and opening remarks
10:00 — 10:50
Charles Manski (Economics, Northwestern University) Identification Problems, Statistical Imprecision, and Medical Decisions under Ambiguity
10:50 — 11:40
Robert Lempert (Pardee RAND Graduate School) Robust Decision Making: Current Practice and Future Directions in Deliberation and Social Choice
11:40 — 12:10
Morning Q&A
12:10 — 14:15
lunch
14:15 — 15:05
Seamus Bradley (Philosophy, Tilburg University) Rational Decisions under Severe Uncertainty: A Philosophical Perspective
15:05 — 15:55
Brian Hill (GREGHEC, CNRS-HEC Paris) Confidence in Beliefs and Rational Decision Making
15:55 — 16:25
coffee
16:25 — 17:15
Marc Fleurbaey (Woodrow Wilson School of Public and International Affairs, Princeton University) Rationality Under Risk and Uncertainty
17:15 — 17:45
Afternoon Q&A
TUESDAY, MAY 23 08:30 — 9:00
coffee
09:00 — 09:50
Anthony O’Hagan (Emeritus, Mathematics and Statistics, University of Sheffield) Eliciting Expert Knowledge and Uncertainty
09:50 — 10:40
Katharine Mach (Earth System Science, Stanford University) Unleashing Expert Judgment in Assessment: IPCC AR5 and Beyond
10:40 — 11:10
coffee
11:10 — 12:00
David Stainforth (Grantham Research Institute on Climate Change and the Environment, LSE) Decision-Specific Explorations of Climate Response Uncertainties with Complex Models – The Role of Non-Discountable Envelopes
12:00 — 12:30
Morning Q&A
12:30 — 14:30
lunch
1
14:30 — 15:20
Yakov Ben-Haim (Mechanical Engineering, Technion) Innovation, Optimization and their Dilemmas: An Info-Gap Perspective
15:20 — 16:10
Casey Helgeson (GREGHEC, CNRS-HEC Paris) A Comparison of Approaches to Deep Uncertainty: Decision Theory and Decision Support
16:10 — 16:40
coffee
16:40 — 17:30
Christian Gollier (Economics, Toulouse School of Economics) An Economic Evaluation of our Responsibilities Towards Future Generations
17:30 — 18:00
Afternoon Q&A
WEDNESDAY, MAY 24 08:30 — 09:00
coffee
09:00 — 09:50
Richard Bradley (Philosophy, LSE) Decision Making under Model Uncertainty
09:50 — 10:40
Itzhak Gilboa (Economics and Decision Sciences, HEC Paris & Tel-Aviv University) Second-Order Induction, Precedents, and Divergence of Opinions
10:40 — 11:10
coffee
11:10 — 11:40
Morning Q&A
11:40 — 13:00
Round table on challenges for the future
ABSTRACTS Innovation, Optimization and their Dilemmas: An Info-Gap Perspective Yakov Ben-Haim (Mechanical Engineering, Technion — Israel Institute of Technology) The search for ever better outcomes should guide the strategic planner in engineering design, public policy, international relations, economics, medical decisions and many other areas of human endeavor. However, uncertainty, ignorance, and surprise may jeopardize the achievement of optimal outcomes. The concept of an innovation dilemma assists in understanding and resolving the planner's challenge. An innovative and highly promising new policy is less familiar than a more standard approach whose implications are more familiar. The innovation, while purportedly better than the standard approach, may be much worse due to uncertainty about the innovation. The resolution (never unambiguous) of the dilemma results from analysis of robustness to surprise (related to resilience, redundancy, flexibility, etc.) and is based on info-gap decision theory. Infogap theory provides decision-support tools for managing the challenges of planning and decision under severe uncertainty. We discuss the method of robustly satisfying critical requirements as a tool for protecting against pernicious uncertainty. We also describe the method of opportune windfalling as a tool for exploiting propitious uncertainty. These ideas will be illustrated by considering two examples: policy formulation for timely recovery from catastrophe, and the amelioration of rural poverty.
Decision Making under Model Uncertainty Richard Bradley (Philosophy, London School of Economics) How should policy planning and decision making accommodate uncertainty about the predictions yielded by scientific models? And how should scientists communicate the uncertainty they have about their own model outputs to policy makers. In this talk I will assess three (classes of) answers to these questions respectively based on forms of model averaging, forms of robustness analysis and forms of confidence grading, drawing on the case of probabilistic hazard assessment and its integration within emergency response and land use planning.
Rational Decisions under Severe Uncertainty: A Philosophical Perspective Seamus Bradley (Philosophy, Tilburg University) Philosophers are interested in rational belief and decision. The standard view is that a rational agent's degrees of belief ought to be probabilities, and that decisions ought to be made by maximising expected utility. There is a growing minority of people who find this picture inadequate. One increasingly popular alternative view -- in philosophy as in many other disciplines -- is the "imprecise probability" (IP) view. This
2
takes an agent's rational degrees of belief to be represented by a set of probability functions rather than a single one. What rational decision ought to consist in this framework is still unclear. Especially when it comes to sequences of decisions, the IP view appears to have some problems. I shall present a problematic example, discuss some proposed decision rules for IP and outline their inadequacies. I shall then suggest a new sort of decision rule which mitigates but doesn't really solve the problem. Finally, I shall argue that there's reason to think that we shouldn't expect more than a mitigation of this problem: in short, we should expect decision making under severe uncertainty to be somewhat under-constrained by principles of rationality.
Rationality under risk and uncertainty Marc Fleurbaey (Woodrow Wilson School of Public and International Affairs, Princeton University) What is rational to do for a decision-maker who is not able to identify states of the world or final consequences, and is not able to assign probabilities to all the relevant events? A set of dominance conditions deriving from stochastic dominance, formulated in a model in which the objects of preferences are act-event pairs, implies expected utility and the use of a well-defined probability measure compatible with the decision-maker’s beliefs. As in Skiadas (1997b), the model is flexible enough to incorporate feelings such as ambiguity aversion in the evaluation of consequences. However, there is a sense in which ambiguity aversion is acceptable for individuals but appears unreasonable (which is not the same as irrational) for policy-makers.
Second-Order Induction, Precedents, and Divergence of Opinions Itzhak Gilboa (Economics and Decision Sciences, HEC Paris & Tel-Aviv University) (joint paper with Rossella Argenziano) Agents make predictions based on similar past cases. The notion of similarity is itself learnt from experience by "second-order induction": past cases also inform agents about the relative importance of various attributes in judging similarity. Second-order induction has several implications. It can explain the importance of precedents, above and beyond their effect on empirical frequencies: a precedent changes the way similarity is judged. Moreover, second order induction may lead to multiple "optimal" similarity functions for explaining past data. As a result, it may lead rational agents who base their beliefs on the same observations to end up with different probabilistic beliefs.
An economic evaluation of our responsibilities towards future generations Christian Gollier (Economics, Toulouse School of Economics) Are we collectively too selfish, acting insufficiently for the well-being of future generations, or are we, on the contrary, too virtuous and too long-termist? Operationally, the myriad of associated decisions are decentralized through price signals (interest rates and risk premia) expressing the way our Society values investment projects, long-term saving products, long-dated assets, and more generally any action that transfers consumption and ecological services across generations. The main objective of this presentation is to discuss the operational tools associated to the problem of valuing the economic, financial, social and environmental impacts of our actions in favor of the distant future. We will explore valuation models that encompass the modern theory of finance, which has a limited normative foundation and which does not account of the deep, non-normal uncertainties surrounding future generations.
A Comparison of Approaches to Deep Uncertainty: Decision Theory and Decision Support Casey Helgeson (GREGHEC, CNRS-HEC Paris) (joint work with Brian Hill) Researchers and practitioners in a variety of fields are developing different approaches to making good decisions under severe uncertainty. It is not always clear how methods in one field relate to those in another. One common element across disciplines is the notion of robustness as a response to deep uncertainty. Maxmin Expected Utility theory and Confidence-Based Decision (developed and discussed in economics and philosophy) and methods such as Robust Decision Making, and Info-Gap (rooted in decision analysis, operational research, and engineering safety and reliability) can all be qualified as ‘robust’. Here we examine the notions of robustness involved in these four approaches.
Confidence in Beliefs and Rational Decision Making Brian Hill (GREGHEC, CNRS-HEC Paris) The standard representation of beliefs, by probability measures, is incapable of representing an agent's confidence in his beliefs. However, as shall be argued in this talk, the agent's confidence in his beliefs plays,
3
and should play, a central role in many of the most difficult decisions which we find ourselves faced with. For instance, in many currently topical decisions requiring scientific expertise, such as environmental policy making, science perhaps does not provide us with as much confidence as we might have liked in our most informed beliefs concerning the relevant issues. Understanding and making such decisions would therefore require a decision theory that departs from the (standard) Bayesian account, and in particular a theory that incorporates the decision maker’s confidence in his beliefs. In this talk, we present parts of an on-going project which aims to formulate and defend a representation of agents' belief states and a normatively reasonable theory of decision which recognises and incorporates confidence in belief. Such a theory can provide a guide for the sorts of decisions just mentioned. We briefly discuss some of its consequences for real-life decision making, particularly in the face of severe uncertainty.
Robust Decision Making: Current Practice and Future Directions in Deliberation and Social Choice Robert Lempert (Pardee RAND Graduate School) In recent years, increasingly good quantitative methods and tools have become available for applying evidence-based analysis to wicked problems: those poorly bounded, framed differently by various groups, and with irreducible uncertainty. In particular, multi-objective robust decision making can now in principle solve the general multi-objective, multi-scenario decision problem under conditions of deep uncertainty, by presenting Pareto Satisfycing surfaces that identify the set of choices that balance among multiple objectives over a wide range of plausible future scenarios. These tools are designed to be used with stakeholders in an iterative process of “deliberation with analysis” that can explore multiple problem framings, help identify promising solutions, present key tradeoffs, and guide participants towards evidence-based consensus solutions. These tools have a solid foundation in the decision sciences as does the fundamental concept of “deliberation with analysis." Nonetheless, the design choice for constructing deliberative processes in any particular decision context remains ad hoc, as does the configuration of the analytic tools used to support it. This talk will first review the theoretical foundations, current methods and tools, and some applications of multi-objective robust decision making. The talk will then describe some current explorations into how these approaches can engage with anthropological, political science, and philosophical understandings of effective processes of deliberation and social choice.
Unleashing expert judgment in assessment: IPCC AR5 and beyond Katharine J. Mach (Earth System Science, Stanford University) In integrative assessment processes, experts evaluate accumulated knowledge and its limits, informing decision-making. Expert judgment is integral to assessment. It is used in analyzing the strengths, weaknesses, and uncertainties of different evidence lines, including limits to inferences from numerical results. More broadly, expert judgment is applied in answering policy-relevant questions and communicating the state of knowledge. In this presentation, I describe advances and challenges in approaches to expert judgment in the Intergovernmental Panel on Climate Change’s Fifth Assessment Report (IPCC AR5). Revised guidance for author teams improved development of balanced judgments on evidence across disciplines. Throughout the report, characterized judgments are more extensive, transparent, and consistent as revealed in analysis of designated degree-of-certainty terminology. Other multi-criteria frameworks complemented this expert-judgment guidance, for example for key risks and reasons for concern. Challenges in developing assessment conclusions persisted, however, especially for findings with substantial uncertainties or normative implications. Based on described AR5 experiences, I suggest a simpler, more rigorous framework for expert judgment in assessment. I also reflect on practices for reducing biases, advancing integration of evidence and expert judgment, and proactively addressing subjective dimensions of expert opinion.
Identification Problems, Statistical Imprecision, and Medical Decisions under Ambiguity Charles F. Manski (Economics, Northwestern University)
When studying collective decision problems, economists have long asked how a planner should act. A standard exercise specifies a set of feasible policies and a welfare function. The planner is presumed to know the welfare achieved by each policy. The objective is to characterize the optimal policy. However, planners often have only partial knowledge of the welfare achieved by alternative policies. Incompleteness of the knowledge obtained in empirical research may stem from both identification problems and statistical imprecision. Given partial knowledge, planners may not be able to determine optimal policies. Instead, they may face decisions under ambiguity (deep uncertainty). This presentation summarizes my recent research on a range of such decision problems when the planner chooses health policy for a population or is a clinician treating individual patients.
4
Eliciting Expert Knowledge and Uncertainty Anthony O’Hagan (Emeritus, Mathematics and Statistics, University of Sheffield) The focus of this talk is on the use of expert judgement to quantify the uncertainty about one or more quantities in the form of a probability distribution. I will first explain why this is an important topic, and then I will describe formal elicitation techniques that are designed to address the key challenges associated with expert judgement: (1) Research in psychology has highlighted a number of sources of error and bias in judgements of probability. How do we avoid or minimise these? (2) We usually wish to elicit judgements from several experts. How do we combine their judgements into a single expression of uncertainty, and what does such a synthesis mean? How many experts should we use and how should we select them?
Decision-Specific Explorations of Climate Response Uncertainties with Complex Models – The Role of Non-Discountable Envelopes David Stainforth (Grantham Research Institute on Climate Change and the Environment, LSE) Complicated atmosphere/ocean global circulation models (AOGCMs) and Earth System Models (ESMs) are widely used to study the consequences of anthropogenic climate change under scenarios of future greenhouse gas emissions. These models are a mixture of reductionist approaches to simulating the well understood physical behaviour of the fluids involved, and physically-inspired statistical relationships to represent processes which take place on scales too small to resolve. In the jargon of climate modelling the latter are referred to as “parameterisations”; they often represent processes which are not as well understood. A nonlinear dynamical systems perspective raises the prospect that small errors in the representation of “small scale”, parameterised processes could have large consequences for probability predictions (the Hawkmoth effect). This has important implications for how we approach extraplolatory predictions of complex systems in general, and particular relevance to the detailed predictions of climate change that are used in impact assessments and adaptation decision making. Approaches to quantifying these uncertainties in the response of the climate system at decision-relevant scales often involve the use of multi-model ensembles and perturbed physics ensembles. Here I will discuss the challenges in interpreting such ensembles and the impact of such challenges on how we might best go about designing these ensembles for scientific research and decision support. The value of purpose-specific design will be emphasised.
5
