Experience is a great teacher . . . except when it isn't. In this groundbreaking guide, learn how the past can deceive and limit us -- and how healthy skepticism can build a better world. Our personal experience is key to who we are and what we do. We judge others by their experience and are judged by ours. Society venerates experience. From doctors to teachers to managers to presidents, the more experience the better. It's not surprising then, that we often fall back on experience when making decisions, an easy way to make judgements about the future, a constant teacher that provides clear lessons. Yet, this intuitive reliance on experience is misplaced. In The Myth of Experience, behavioral scientists Emre Soyer and Robin Hogarth take a transformative look at experience and the many ways it deceives and misleads us. From distorting the past to limiting creativity to reducing happiness, experience can cause misperceptions and then reinforce them without our awareness. Instead, the authors argue for a nuanced approach, where a healthy skepticism toward the lessons of experience results in more reliable decisions and sustainable growth. Soyer and Hogarth illustrate the flaws of experience -- with real-life examples from bloodletting to personal computers to pandemics -- and distill cutting-edge research as a guide to decision-making, as well as provide the remedies needed to improve our judgments and choices in the workplace and beyond.

"A Critical Decision Method (CDM) has been developed for knowledge elicitation. The CDM, an extension of the critical incident technique, includes protocol analysis and memory recall tasks to study cognitive performance. A set of probes is employed to trace the development of situation assessment during critical incidents, and to determine the decision strategies used. The outputs of the method include inventories of the critical cues, graphic portrayals of the situation assessment process, and categorization of the decision strategies. Thus far, the method has been used with a variety of decisions and appears especially well suited to studying cognitive performance in naturalistic settings. It also appears valuable for addressing the highly skilled decision maker, and for eliciting the analytical and perceptual bases of proficient performance. Applications have been made for training, decision support systems, and the development and evaluation of knowledge based systems." -- Abstract.

This book is about elicitation: the facilitation of the quantitative expression of subjective judgement about matters of fact, interacting with subject experts, or about matters of value, interacting with decision makers or stakeholders. It offers an integrated presentation of procedures and processes that allow analysts and experts to think clearly about numbers, particularly the inputs for decision support systems and models. This presentation encompasses research originating in the communities of structured probability elicitation/calibration and multi-criteria decision analysis, often unaware of each other’s developments. Chapters 2 through 9 focus on processes to elicit uncertainty from experts, including the Classical Method for aggregating judgements from multiple experts concerning probability distributions; the issue of validation in the Classical Method; the Sheffield elicitation framework; the IDEA protocol; approaches following the Bayesian perspective; the main elements of structured expert processes for dependence elicitation; and how mathematical methods can incorporate correlations between experts. Chapters 10 through 14 focus on processes to elicit preferences from stakeholders or decision makers, including two chapters on problems under uncertainty (utility functions), and three chapters that address elicitation of preferences independently of, or in absence of, any uncertainty elicitation (value functions and ELECTRE). Two chapters then focus on cross-cutting issues for elicitation of uncertainties and elicitation of preferences: biases and selection of experts. Finally, the last group of chapters illustrates how some of the presented approaches are applied in practice, including a food security case in the UK; expert elicitation in health care decision making; an expert judgement based method to elicit nuclear threat risks in US ports; risk assessment in a pulp and paper manufacturer in the Nordic countries; and elicitation of preferences for crop planning in a Greek region.

The disciplines of knowledge engineering and knowledge management are closely tied. Knowledge engineering deals with the development of information systems in which knowledge and reasoning play pivotal roles. Knowledge management, a newly developed field at the intersection of computer science and management, deals with knowledge as a key resource in modern organizations. Managing knowledge within an organization is inconceivable without the use of advanced information systems; the design and implementation of such systems pose great organization as well as technical challenges.

Elicitation is the process of extracting expert knowledge about some unknown quantity or quantities, and formulating that information as a probability distribution. Elicitation is important in situations, such as modelling the safety of nuclear installations or assessing the risk of terrorist attacks, where expert knowledge is essentially the only source of good information. It also plays a major role in other contexts by augmenting scarce observational data, through the use of Bayesian statistical methods. However, elicitation is not a simple task, and practitioners need to be aware of a wide range of research findings in order to elicit expert judgements accurately and reliably. Uncertain Judgements introduces the area, before guiding the reader through the study of appropriate elicitation methods, illustrated by a variety of multi-disciplinary examples. This is achieved by: Presenting a methodological framework for the elicitation of expert knowledge incorporating findings from both statistical and psychological research. Detailing techniques for the elicitation of a wide range of standard distributions, appropriate to the most common types of quantities. Providing a comprehensive review of the available literature and pointing to the best practice methods and future research needs. Using examples from many disciplines, including statistics, psychology, engineering and health sciences. Including an extensive glossary of statistical and psychological terms. An ideal source and guide for statisticians and psychologists with interests in expert judgement or practical applications of Bayesian analysis, Uncertain Judgements will also benefit decision-makers, risk analysts, engineers and researchers in the medical and social sciences.