Unexpected events during an evaluation all too often send evaluators into crisis mode. This insightful book provides a systematic framework for diagnosing, anticipating, accommodating, and reining in costs of evaluation surprises. The result is evaluation that is better from a methodological point of view, and more responsive to stakeholders. Jonathan A. Morell identifies the types of surprises that arise at different stages of a program's life cycle and that may affect different aspects of the evaluation, from stakeholder relationships to data quality, methodology, funding, deadlines, information use, and program outcomes. His analysis draws on 18 concise cases from well-known researchers in a variety of evaluation settings. Morell offers guidelines for responding effectively to surprises and for determining the risks and benefits of potential solutions.

WIDTH: 405pt; BORDER-COLLAPSE: collapse border=0 cellSpacing=0 cellPadding=0 width=540> WIDTH: 405pt; mso-width-source: userset; mso-width-alt: 19748 width=540> HEIGHT: 31.5pt height=42> BORDER-BOTTOM: #f0f0f0; BORDER-LEFT: #f0f0f0; BACKGROUND-COLOR: transparent; WIDTH: 405pt; HEIGHT: 31.5pt; BORDER-TOP: #f0f0f0; BORDER-RIGHT: #f0f0f0 class=xl65 height=42 width=540>GSP 229 contains 54 papers on risk and uncertainty in foundation engineering presented in honor of Fred H. Kulhawy.

In Changing Bureaucracies, international experts provide an unparalleled look at how public sector bureaucracies can better adapt to the reality of unprecedented levels of uncertainty and complexity, and how they can better respond to the emerging needs and demands of citizens and beneficiaries. In particular, they discuss in detail how evaluation can play an important role in aiding bureaucracies in adapting, while noting that the value of evaluation is not at all automatic. Written in a clear and accessible prose, the contributors identify stability as a strength of bureaucratic structures, although adaptability is required in order to remain relevant. They also emphasize the need for bureaucratic rules and practices to be open to examination, such as through evaluation, noting that these rules may take on a life of their own, increasing distrust and conflicting with a meaningful focus on how outcomes and impacts benefit citizens. The book concludes with guidance for both evaluators and for public sector leaders about steps that they can take to improve the responsiveness and relevance of public sector organizations. Pioneering the provision of reflections on how evaluation can play an important role in aiding bureaucracies in adapting, Changing Bureaucracies is an important acquisition for public sector leaders, evaluators, evaluation managers and commissioners and academics alike.

The public depends on competent risk assessment from the federal government and the scientific community to grapple with the threat of pollution. When risk reports turn out to be overblownâ€"or when risks are overlookedâ€"public skepticism abounds. This comprehensive and readable book explores how the U.S. Environmental Protection Agency (EPA) can improve its risk assessment practices, with a focus on implementation of the 1990 Clean Air Act Amendments. With a wealth of detailed information, pertinent examples, and revealing analysis, the volume explores the "default option" and other basic concepts. It offers two views of EPA operations: The first examines how EPA currently assesses exposure to hazardous air pollutants, evaluates the toxicity of a substance, and characterizes the risk to the public. The second, more holistic, view explores how EPA can improve in several critical areas of risk assessment by focusing on cross-cutting themes and incorporating more scientific judgment. This comprehensive volume will be important to the EPA and other agencies, risk managers, environmental advocates, scientists, faculty, students, and concerned individuals.

This open access book focuses on both the theory and practice associated with the tools and approaches for decisionmaking in the face of deep uncertainty. It explores approaches and tools supporting the design of strategic plans under deep uncertainty, and their testing in the real world, including barriers and enablers for their use in practice. The book broadens traditional approaches and tools to include the analysis of actors and networks related to the problem at hand. It also shows how lessons learned in the application process can be used to improve the approaches and tools used in the design process. The book offers guidance in identifying and applying appropriate approaches and tools to design plans, as well as advice on implementing these plans in the real world. For decisionmakers and practitioners, the book includes realistic examples and practical guidelines that should help them understand what decisionmaking under deep uncertainty is and how it may be of assistance to them. Decision Making under Deep Uncertainty: From Theory to Practice is divided into four parts. Part I presents five approaches for designing strategic plans under deep uncertainty: Robust Decision Making, Dynamic Adaptive Planning, Dynamic Adaptive Policy Pathways, Info-Gap Decision Theory, and Engineering Options Analysis. Each approach is worked out in terms of its theoretical foundations, methodological steps to follow when using the approach, latest methodological insights, and challenges for improvement. In Part II, applications of each of these approaches are presented. Based on recent case studies, the practical implications of applying each approach are discussed in depth. Part III focuses on using the approaches and tools in real-world contexts, based on insights from real-world cases. Part IV contains conclusions and a synthesis of the lessons that can be drawn for designing, applying, and implementing strategic plans under deep uncertainty, as well as recommendations for future work. The publication of this book has been funded by the Radboud University, the RAND Corporation, Delft University of Technology, and Deltares.

A must-read for anyone who makes business decisions that have a major financial impact. As the recent collapse on Wall Street shows, we are often ill-equipped to deal with uncertainty and risk. Yet every day we base our personal and business plans on uncertainties, whether they be next month’s sales, next year’s costs, or tomorrow’s stock price. In The Flaw of Averages, Sam Savageknown for his creative exposition of difficult subjects describes common avoidable mistakes in assessing risk in the face of uncertainty. Along the way, he shows why plans based on average assumptions are wrong, on average, in areas as diverse as healthcare, accounting, the War on Terror, and climate change. In his chapter on Sex and the Central Limit Theorem, he bravely grasps the literary third rail of gender differences. Instead of statistical jargon, Savage presents complex concepts in plain English. In addition, a tightly integrated web site contains numerous animations and simulations to further connect the seat of the reader’s intellect to the seat of their pants. The Flaw of Averages typically results when someone plugs a single number into a spreadsheet to represent an uncertain future quantity. Savage finishes the book with a discussion of the emerging field of Probability Management, which cures this problem though a new technology that can pack thousands of numbers into a single spreadsheet cell. Praise for The Flaw of Averages “Statistical uncertainties are pervasive in decisions we make every day in business, government, and our personal lives. Sam Savage’s lively and engaging book gives any interested reader the insight and the tools to deal effectively with those uncertainties. I highly recommend The Flaw of Averages.” —William J. Perry, Former U.S. Secretary of Defense “Enterprise analysis under uncertainty has long been an academic ideal. . . . In this profound and entertaining book, Professor Savage shows how to make all this practical, practicable, and comprehensible.” —Harry Markowitz, Nobel Laureate in Economics

The U.S. Environmental Protection Agency (EPA) is one of several federal agencies responsible for protecting Americans against significant risks to human health and the environment. As part of that mission, EPA estimates the nature, magnitude, and likelihood of risks to human health and the environment; identifies the potential regulatory actions that will mitigate those risks and protect public health1 and the environment; and uses that information to decide on appropriate regulatory action. Uncertainties, both qualitative and quantitative, in the data and analyses on which these decisions are based enter into the process at each step. As a result, the informed identification and use of the uncertainties inherent in the process is an essential feature of environmental decision making. EPA requested that the Institute of Medicine (IOM) convene a committee to provide guidance to its decision makers and their partners in states and localities on approaches to managing risk in different contexts when uncertainty is present. It also sought guidance on how information on uncertainty should be presented to help risk managers make sound decisions and to increase transparency in its communications with the public about those decisions. Given that its charge is not limited to human health risk assessment and includes broad questions about managing risks and decision making, in this report the committee examines the analysis of uncertainty in those other areas in addition to human health risks. Environmental Decisions in the Face of Uncertainty explains the statement of task and summarizes the findings of the committee.