Jan H. Kwakkel

Classifying and communicating uncertainties in model-based policy analysis

Uncertainty is of paramount importance in modern day decision making. In response to this, it has been suggested that policy analysts have to be more careful in communicating the uncertainties that are inherent in policy advice. To support policy analysts in identifying uncertainties and communicating these uncertainties to decision makers, an uncertainty matrix was proposed by Walker et al. (2003), which synthesised various taxonomies, frameworks and typologies of uncertainties from different decision support fields. Since its publication, this framework has been applied to different domains. As a result, the framework has undergone changes resulting in a proliferation of uncertainty frameworks. This proliferation runs counter to the purpose of the original framework. This paper presents an extensive review of the literature that builds on Walker et al. (2003). In light of this, a synthesis is presented, which can be used to assess and communicate uncertainties in model-based policy analysis studies.

Developing dynamic adaptive policy pathways: a computer-assisted approach for developing adaptive strategies for a deeply uncertain world

Sustainable water management in a changing environment full of uncertainty is profoundly challenging. To deal with these uncertainties, dynamic adaptive policies that can be changed over time are suggested. This paper presents a model-driven approach supporting the development of promising adaptation pathways, and illustrates the approach using a hypothetical case. We use robust optimization over uncertainties related to climate change, land use, cause-effect relations, and policy efficacy, to identify the most promising pathways. For this purpose, we generate an ensemble of possible futures and evaluate candidate pathways over this ensemble using an Integrated Assessment Meta Model. We understand ‘most promising’ in terms of the robustness of the performance of the candidate pathways on multiple objectives, and use a multi-objective evolutionary algorithm to find the set of most promising pathways. This results in an adaptation map showing the set of most promising adaptation pathways and options for transferring from one pathway to another. Given the pathways and signposts, decision-makers can make an informed decision on a dynamic adaptive plan in a changing environment that is able to achieve their intended objectives despite the myriad of uncertainties.

An exploratory approach for adaptive policymaking by using multi-objective robust optimization

Abstract Developing robust policies for complex systems is a profound challenge because of their nonlinear and unpredictable nature. Dealing with these characteristics requires innovative approaches. A possible approach is to design policies that can be adapted over time in response to how the future unfolds. An essential part of adaptive policymaking is specifying under what conditions, and in which way, to adapt the policy. The performance of an adaptive policy is critically dependent on this: if the policy is adapted too late or too early, significant deterioration in policy performance can be incurred. An additional complicating factor is that in almost any policy problem, a multiplicity of divergent and potentially conflicting objectives has to be considered. In this paper we tackle both problems simultaneously through the use of multi-objective robust simulation optimization. Robust optimization helps in specifying appropriate conditions for adapting a policy, by identifying conditions that produce satisfactory results across a large ensemble of scenarios. Multi-objective optimization helps in identifying such conditions for a set of criteria, and providing insights into the tradeoffs between these criteria. Simulation is used for evaluating policy performance. This approach results in the identification of multiple alternative conditions under which to adapt a policy, rather than a single set of conditions. This creates the possibility of an informed policy debate on trade-offs. The approach is illustrated through a case study on designing a robust policy for supporting the transition toward renewable energy systems in the European Union. The results indicate that the proposed approach can be efficiently used for developing policy suggestions and for improving decision support for policymakers. By extension, it is possible to apply this methodology in dynamically complex and deeply uncertain systems such as public health, financial systems, transportation, and housing.

Coping with the wickedness of public policy problems: Approaches for decision-making under deep uncertainty

In many planning problems, planners face major challenges in coping with uncertain and changing physical conditions, and rapid unpredictable socioeconomic development. How should society prepare itself for this confluence of uncertainty? Given the presence of irreducible uncertainties, there is no straightforward answer to this question. Effective decisions must be made under unavoidable uncertainty (Dessai et al. 2009; Lempert et al. 2003). In recent years, this has been labeled as decision making under deep uncertainty. Deep uncertainty means that the various parties to a decision do not know or cannot agree on the system and its boundaries; the outcomes of interest and their relative importance; the prior probability distribution for uncertain inputs to the system (Lempert et al. 2003; Walker et al. 2013); or decisions are made over time in dynamic interaction with the system and cannot be considered independently (Haasnoot et al. 2013a, b; Hallegatte et al. 2012). From a decision analytic point of view, this implies that there are a large number of plausible alternative models, alternative sets of weights to assign to the different outcomes of interest, different sets of inputs for the uncertain model parameters, and different (sequences of) candidate solutions (Kwakkel et al. 2010). Decision making under deep uncertainty is a particular type of wicked problem (Rittel and Webber 1973). Wicked problems are problems characterized by the involvement of a variety of stakeholders and decision makers with conflicting values and diverging ideas for solutions (Churchman 1967). What makes wicked problems especially pernicious is that even the problem formulation itself is contested (Rittel and Webber 1973). System analytic approaches presuppose a separation between the problem formulation and the solution. In wicked problem situations this distinction breaks down. Solutions and problem formulation are intertwined with each other. Depending on how a problem is framed, alternative solutions come to the fore; and, vice versa, depending on the available or preferred solutions, the problem can be framed differently. Even if there is agreement on the difference between observed and desired outcomes, rival explanations for the existence of this difference are available, and, hence, different solutions can be preferred. An additional factor adding to the wickedness is that decision makers can ill afford to be wrong. The consequences of any decision on wicked problems can be profound, difficult if not impossible to reverse, and result in lock-ins for future decision making. Planning and decision making in wicked problem situations should, therefore, be understood as an argumentative process: in which the problem formulation, a shared understanding of system functioning and how this gives rise to the problem, and the set of promising solutions, emerge gradually through debate among the involved decision makers and stakeholders (Dewulf et al. 2005). When even the problem formulation itself is uncertain and contested, planning and decision making requires an iterative approach that facilitates learning across alternative framings of the problem, and learning about stakeholder preferences and trade-offs, all in pursuit of a collaborative process of discovering what is possible (Herman et al. 2015). Modeling and optimization can play a role in facilitating this learning. They can help in discovering a set of possible actions that is worth closer inspection, and make the trade-offs among these actions more transparent (Liebman 1976; Reed and Kasprzyk 2009). Under the moniker of decision making under deep uncertainty, a variety of new approaches and tools are being put forward. Emerging approaches include (multiobjective) robust decision making (Kasprzyk et al. 2013; Lempert et al. 2006), info-gap decision theory (Ben Haim 2001), dynamic adaptive policy pathways (Haasnoot et al. 2013a, b), and decision scaling (Brown et al. 2012). A common feature of these approaches is that they are exploratory model-based strategies for designing adaptive and robust plans or policies. Although these frameworks are used in a wide variety of applications, they have been most commonly applied in the water domain, in which climate change and social change are key concerns that affect the long-term viability of current management plans and strategies. Liebman (1976) recognized that water resources planning problems are wicked problems in which modeling, simulation, and optimization cannot be straightforwardly applied. In recent years, this observation has been reiterated (Herman et al. 2015; Lund 2012; Reed and Kasprzyk 2009). If decision making under deep uncertainty is a particular type of wicked problem, to what extent do the recent methodological advances address some of the key aspects of what makes wicked problems wicked? To answer this question, the authors look at two exemplary approaches for supporting decision making under deep uncertainty: (multiobjective) robust decision making and dynamic adaptive policy pathways. This article first briefly outlines each approach, and then discusses some of the ongoing scientific work aimed at integrating the two approaches. This sets the stage for a critical discussion of these approaches and how they touch on the key concerns of supporting decision making in wicked problem situations.

Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways

Exploring adaptation pathways is an emerging approach for supporting decision making under uncertain changing conditions. An adaptation pathway is a sequence of policy actions to reach specified objectives. To develop adaptation pathways, interactions between environment and policy response need to be analysed over time for an ensemble of plausible futures. A fast, integrated model can facilitate this. Here, we describe the development and evaluation of such a model, an Integrated Assessment Metamodel (IAMM), to explore adaptation pathways in the Rhine delta for a decision problem currently faced by the Dutch Government. The theory-motivated metamodel is a simplified physically based model. Closed questions reflecting the required accuracy were used to evaluate the models fitness. The results show that such a model fits the purpose of screening and ranking of policy options and pathways to support the strategic decision making. A complex model can subsequently be used to obtain more detailed information.

Comparing Robust Decision-Making and Dynamic Adaptive Policy Pathways for model-based decision support under deep uncertainty

A variety of model-based approaches for supporting decision-making under deep uncertainty have been suggested, but they are rarely compared and contrasted. In this paper, we compare Robust Decision-Making with Dynamic Adaptive Policy Pathways. We apply both to a hypothetical case inspired by a river reach in the Rhine Delta of the Netherlands, and compare them with respect to the required tooling, the resulting decision relevant insights, and the resulting plans. The results indicate that the two approaches are complementary. Robust Decision-Making offers insights into conditions under which problems occur, and makes trade-offs transparent. The Dynamic Adaptive Policy Pathways approach emphasizes dynamic adaptation over time, and thus offers a natural way for handling the vulnerabilities identified through Robust Decision-Making. The application also makes clear that the analytical process of Robust Decision-Making is path-dependent and open ended: an analyst has to make many choices, for which Robust Decision-Making offers no direct guidance. This paper compares Robust Decision-Making (RDM) and Dynamic Adaptive Policy Pathways (DAPP).RDM and DAPP have different merits, which highlight their complementarity.RDM has a clear analytical process and the application is reasonably straight forward.DAPP offers a convenient framework for designing plans for dynamic adaptation over time.

Improving scenario discovery for handling heterogeneous uncertainties and multinomial classified outcomes

Scenario discovery is a novel model-based approach to scenario development in the presence of deep uncertainty. Scenario discovery frequently relies on the Patient Rule Induction Method (PRIM). PRIM identifies regions in the model input space that are highly predictive of producing model outcomes that are of interest. To identify these, PRIM uses a lenient hill climbing optimization procedure. PRIM struggles when confronted with cases where the uncertain factors are a mix of data types, and can be used only for binary classifications. We compare two more lenient objective functions which both address the first problem, and an alternative objective function using Gini impurity which addresses the second problem. We assess the efficacy of the modification using previously published cases. Both modifications are effective. The more lenient objective functions produce better descriptions of the data, while the Gini impurity objective function allows PRIM to be used when handling multinomial classified data. We compare three objective functions for PRIM in case of binary classified data.The more lenient objective functions outperform the less lenient objective functions.We introduce a new objective function for PRIM in case of multinomial classified data.We compare PRIM with the multinomial objective function to both CART, and sequential use of PRIM on each class separately.

Assessing the efficacy of dynamic adaptive planning of infrastructure: results from computational experiments

In this paper we assess the efficacy of a dynamic adaptive planning (DAP) approach for guiding the long-term development of infrastructure. The efficacy of the approach is tested on the specific case of airport strategic planning. Utilizing a fast and simple model of an airport, and a composition of small models that can generate a wide spectrum of alternative futures, the performance of a dynamic adaptive plan is compared with the performance of a static, rigid implementation plan across a wide spectrum of conceivable futures. These computational experiments reveal that the static rigid plan outperforms the dynamic adaptive plan in only a small part of the spectrum. Moreover, given the wide array of possible futures, the dynamic adaptive plan has a narrower spread of outcomes than the static rigid plan, implying that the dynamic adaptive plan exposes planners to less uncertainty about its future performance despite the wide variety of uncertainties that are present. These computational results confirm theoretical hypotheses in the literature that DAP approaches are more efficacious for planning under uncertainty.

Uncertainty in the Framework of Policy Analysis

Synonyms: uncertainty, doubt, dubiety, skepticism, suspicion, mistrust, mean lack of sureness about someone or something. Uncertainty may range from falling short of certainty to an almost complete lack of conviction or knowledge especially about an outcome or result. Doubt suggests both uncertainty and inability to make a decision. Dubiety stresses a wavering between conclusions. Skepticism implies unwillingness to believe without conclusive evidence. Suspicion stresses lack of faith in the truth, reality, fairness, or reliability of something or someone. Mistrust implies a genuine doubt based upon suspicion. [Merriam-Webster Online Dictionary].

Dealing with Uncertainties in Fresh Water Supply: Experiences in the Netherlands

Developing fresh water supply strategies for the long term needs to take into account the fact that the future is deeply uncertain. Not only the extent of climate change and the extent and nature of its impacts are unknown, also socio-economic conditions may change in unpredictable ways, as well as social preferences. Often, it is not possible to find solid ground for estimating probabilities for the relevant range of imaginable possible future developments. Yet, some of these may have profound impacts and consequences for society which could be reduced by timely proactive adaptation. In response to these and similar challenges, various approaches, methods and techniques have been proposed and are being developed to specifically address long-term strategy development under so-called deep uncertainty. This paper, first, offers a brief overview of developments in the field of planning under (deep) uncertainty. Next, we illustrate application of three different approaches to fresh water provision planning under uncertainty in case studies in the Netherlands: a resilience approach, oriented to (re) designing fresh water systems in such a way that they will be less vulnerable, resp. will be able to recover easily from future disturbances; a robustness approach, oriented to quantitative assessment of system performance for various system configurations (adaptation options) under a range of external disturbances, and an exploratory modeling approach, developed to explore policy effectiveness and system operation under a very wide set of assumptions about future conditions.