Table of Contents

Realistic Evaluation shows how program evaluation needs to be, and can be bettered. It presents a profound yet highly readable critique of current evaluation practice, and goes on to introduce a `manifesto′ and `handbook′ for a fresh approach. The main body of this book is devoted to the articulation of a new evaluation paradigm, which promises greater validity and utility from the findings of evaluation studies. The authors call this new approach `realistic evaluation′. The name reflects the paradigm′s foundation in scientific realist philosophy, its commitment to the idea that programmes deal with real problems rather than mere social constructions, and its primary intention, which is to inform realistic developments in policy making that benefit programme participants and the public. Ray Pawson and Nicholas Tilley argue with passion that scientific evaluation requires a careful blend of theory and method, quality and quantity, ambition and realism. The book offers a complete blueprint for evaluation activities, running from design to data collection and analysis to the cumulation of findings across programmes and onto the realization of research into policy. The argument is developed using practical examples throughout and is grounded in the major fields of programme evaluation. This book will be essential reading for all those involved in the evaluation process especially those researchers, students and practitioners in the core disciplines of sociology, social policy, criminology, health and education. `This book is a must for those engaged in the field, providing a fully illustrated text on evaluation with numerous examples from the criminal justice system. Unusually, it offers something for the academic, practitioner and student alike. I found Pawson and Tilley′s latest work on evaluation an enjoyable and informative read. For myself their "realistic evaluation" clarified and formalised a jumbled set of ideas I had already been developing. Although not everyone will agree with the methodology proposed by the authors, this book is a valuable read as it will cause most of us at least to review our methodological stance′ - International Journal of Police Science and Management `This is an engaging book with a strong sense of voice and communicative task. The voice is sometimes strident, but always clear. Its communicative qualities are evident equally in its structure: lots of signposting for the reader within and across chapters′ - Language Teaching Research `This provocative, elegant and highly insightful book focuses on the effective incorporation of actual practice into the formulation of evaluation methodology. What a pleasure to read sentences like: "The research act involves "learning" a stakeholder′s theories, formalizing them, and "teaching" them back to that informant who is then in a position to comment upon, clarify and further refine the key ideas". Pawson and Tilley have given us a wise, witty and persuasive account of how real practitioner experience might be encouraged to intrude on (and modify) researchers′ concepts about program processes and outcomes. This holds important promise for achieving something that is devoutly to be wished: closer interaction among at least some researchers and some policy makers′ - Eleanor Chelimsky, Past-President of the American Evaluation Association `This is a sustained methodological argument by two wordly-wise social scientists. Unashamedly intellectual, theoretically ambitious yet with a clear but bounded conception of evaluation. It is articulate, occasionally eloquent and always iconoclastic, whilst eschewing "paradigm wars". The Pawson and Tilley "realist" call to arms threatens to take no prisoners among experimentalists, constructivists or pluralists. It is the kind of book that clarifies your thoughts, even when you disagree with everything they say′ - Elliot Stern, The Tavistock Institute

Evaluation researchers are tasked with providing the evidence to guide programme building and to assess its outcomes. As such, they labour under the highest expectations - bringing independence and objectivity to policy making. They face huge challenges, given the complexity of modern interventions and the politicised backdrop to all of their investigations. They have responded with a huge portfolio of research techniques and, through their professional associations, have set up schemes to establish standards for evaluative inquiry and to accredit evaluation practitioners. A big question remains. Has this monumental effort produced a progressive, cumulative and authoritative body of knowledge that we might think of as evaluation science? This is the question addressed by Ray Pawson in this sequel to Realistic Evaluation and Evidence-based Policy. In answer, he provides a detailed blueprint for an evaluation science based on realist principles.

Evaluation for the 21st Century features thoughtfully written introductions to each of the main sections that provide a context and synthesis of the various evaluators' chapters. After reading this groundbreaking book, researchers and practitioners will be able to recognize these new developments in evaluation as they encounter them, place them in context, and incorporate them into their own evaluation professions and practices.

Bringing together leading theorists, researchers and policy makers with expertise in using realist methods, this book is a definitive guide to putting realist methodologies into practice. Not just an overview of the field, this book looks to extend current debates and apply realist methods to new and practical challenges in social research. Featuring practical, worked examples of how to turn theory into evidence, it empowers readers not just to understand realist methods, but to use them. It will help readers: - Negotiate the complexity of relational systems - Understand the importance and relevance of cumulative theory - Address concerns over data sources and quality - Be flexible and creative in realist approaches - Produce useful evidence for policy. Sophisticated and globally minded, this book is the perfect addition to the ongoing development and application of realist methods across evaluation, synthesis, and social research.

In this important new book, Ray Pawson examines the recent spread of evidence-based policy making across the Western world. Few major public initiatives are mounted these days in the absence of a sustained attempt to evaluate them. Programmes are tried, tried and tried again and researched, researched and researched again. And yet it is often difficult to know which interventions, and which inquiries, will withstand the test of time. The evident solution, going by the name of evidence-based policy, is to take the longer view. Rather than relying on one-off studies, it is wiser to look to the ′weight of evidence′. Accordingly, it is now widely agreed the most useful data to support policy decisions will be culled from systematic reviews of all the existing research in particular policy domains. This is the consensual starting point for Ray Pawson′s latest foray into the world of evaluative research. But this is social science after all and harmony prevails only in the first chapter. Thereafter, Pawson presents a devastating critique of the dominant approach to systematic review - namely the ′meta-analytic′ approach as sponsored by the Cochrane and Campbell collaborations. In its place is commended an approach that he terms ′realist synthesis′. On this vision, the real purpose of systematic review is better to understand programme theory, so that policies can be properly targeted and developed to counter an ever-changing landscape of social problems. The book will be essential reading for all those who loved (or loathed) the arguments developed in Realistic Evaluation (Sage, 1997). It offers a complete blueprint for research synthesis, supported by detailed illustrations and worked examples from across the policy waterfront. It will be of especial interest to policy-makers, practitioners, researchers and students working in health, education, employment, social care, criminal justice, regeneration and welfare.

Evaluating Research in Academic Journals is a guide for students who are learning how to evaluate reports of empirical research published in academic journals. It breaks down the process of evaluating a journal article into easy-to-understand steps, and emphasizes the practical aspects of evaluating research – not just how to apply a list of technical terms from textbooks. The book avoids oversimplification in the evaluation process by describing the nuances that may make an article publishable even when it has serious methodological flaws. Students learn when and why certain types of flaws may be tolerated, and why evaluation should not be performed mechanically. Each chapter is organized around evaluation questions. For each question, there is a concise explanation of how to apply it in the evaluation of research reports. Numerous examples from journals in the social and behavioral sciences illustrate the application of the evaluation questions, and demonstrate actual examples of strong and weak features of published reports. Common-sense models for evaluation combined with a lack of jargon make it possible for students to start evaluating research articles the first week of class. New to this edition New chapters on: evaluating mixed methods research evaluating systematic reviews and meta-analyses program evaluation research Updated chapters and appendices that provide more comprehensive information and recent examples Full new online resources: test bank questions and PowerPoint slides for instructors, and self-test chapter quizzes, further readings and additional journal examples for students.