The Agency for Healthcare Research and Quality (AHRQ) commissioned the RTI International–University of North Carolina at Chapel Hill (RTI-UNC) Evidence-based Practice Center (EPC) to explore how systematic review groups have dealt with clinical heterogeneity and to seek out best practices for addressing clinical heterogeneity in systematic reviews (SRs) and comparative effectiveness reviews (CERs). Such best practices, to the extent they exist, may enable AHRQ's EPCs to address critiques from patients, clinicians, policymakers, and other proponents of health care about the extent to which “average” estimates of the benefits and harms of health care interventions apply to individual patients or to small groups of patients sharing similar characteristics. Such users of reviews often assert that EPC reviews typically focus on broad populations and, as a result, often lack information relevant to patient subgroups that are of particular concern to them. More important, even when EPCs evaluate literature on homogeneous groups, there may be varying individual treatment for no apparent reason, indicating that average treatment effect does not point to the best treatment for any given individual. Thus, the health care community is looking for better ways to develop information that may foster better medical care at a “personal” or “individual” level. To address our charge for this methods project, the EPC set out to answer six key questions (KQ). Key questions for methods report on clinical heterogeneity include: 1. What is clinical heterogeneity? a. How has it been defined by various groups? b. How is it distinct from statistical heterogeneity? c. How does it fit with other issues that have been addressed by the AHRQ Methods Manual for CERs? 2. How have systematic reviews dealt with clinical heterogeneity in the key questions? a. What questions have been asked? b. How have they pre-identified population subgroups with common clinical characteristics that modify their intervention-outcome association? c. What are best practices in key questions and how these subgroups have been identified? 3. How have systematic reviews dealt with clinical heterogeneity in the review process? a. What do guidance documents of various systematic review groups recommend? b. How have EPCs handled clinical heterogeneity in their reviews? c. What are best practices in searching for and interpreting results for particular subgroups with common clinical characteristics that may modify their intervention-outcome association? 4. What are critiques in how systematic reviews handle clinical heterogeneity? a. What are critiques from specific reviews (peer and public) on how EPCs handled clinical heterogeneity? b. What general critiques (in the literature) have been made against how systematic reviews handle clinical heterogeneity? 5. What evidence is there to support how to best address clinical heterogeneity in a systematic review? 6. What questions should an EPC work group on clinical heterogeneity address? Heterogeneity (of any type) in EPC reviews is important because its appearance suggests that included studies differed on one or more dimensions such as patient demographics, study designs, coexisting conditions, or other factors. EPCs then need to clarify for clinical and other audiences, collectively referred to as stakeholders, what are the potential causes of the heterogeneity in their results. This will allow the stakeholders to understand whether and to what degree they can apply this information to their own patients or constituents. Of greatest importance for this project was clinical heterogeneity, which we define as the variation in study population characteristics, coexisting conditions, cointerventions, and outcomes evaluated across studies included in an SR or CER that may influence or modify the magnitude of the intervention measure of effect (e.g., odds ratio, risk ratio, risk difference).

This User’s Guide is a resource for investigators and stakeholders who develop and review observational comparative effectiveness research protocols. It explains how to (1) identify key considerations and best practices for research design; (2) build a protocol based on these standards and best practices; and (3) judge the adequacy and completeness of a protocol. Eleven chapters cover all aspects of research design, including: developing study objectives, defining and refining study questions, addressing the heterogeneity of treatment effect, characterizing exposure, selecting a comparator, defining and measuring outcomes, and identifying optimal data sources. Checklists of guidance and key considerations for protocols are provided at the end of each chapter. The User’s Guide was created by researchers affiliated with AHRQ’s Effective Health Care Program, particularly those who participated in AHRQ’s DEcIDE (Developing Evidence to Inform Decisions About Effectiveness) program. Chapters were subject to multiple internal and external independent reviews. More more information, please consult the Agency website: www.effectivehealthcare.ahrq.gov)

Healthcare decision makers in search of reliable information that compares health interventions increasingly turn to systematic reviews for the best summary of the evidence. Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies, and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services. Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, for professional medical societies and other organizations that develop clinical practice guidelines. Too often systematic reviews are of uncertain or poor quality. There are no universally accepted standards for developing systematic reviews leading to variability in how conflicts of interest and biases are handled, how evidence is appraised, and the overall scientific rigor of the process. In Finding What Works in Health Care the Institute of Medicine (IOM) recommends 21 standards for developing high-quality systematic reviews of comparative effectiveness research. The standards address the entire systematic review process from the initial steps of formulating the topic and building the review team to producing a detailed final report that synthesizes what the evidence shows and where knowledge gaps remain. Finding What Works in Health Care also proposes a framework for improving the quality of the science underpinning systematic reviews. This book will serve as a vital resource for both sponsors and producers of systematic reviews of comparative effectiveness research.

Clinical research presents health care providers with information on the natural history and clinical presentations of disease as well as diagnostic and treatment options. In today's healthcare system, patients, physicians, clinicians and family caregivers often lack the sufficient scientific data and evidence they need to determine the best course of treatment for the patients' medical conditions. Initial National Priorities for Comparative Effectiveness Research(CER) is designed to fill this knowledge gap by assisting patients and healthcare providers across diverse settings in making more informed decisions. In this 2009 report, the Institute of Medicine's Committee on Comparative Effectiveness Research Prioritization establishes a working definition of CER, develops a priority list of research topics, and identifies the necessary requirements to support a robust and sustainable CER enterprise. As part of the 2009 American Recovery and Reinvestment Act, Congress appropriated $1.1 billion in federal support of CER, reflecting legislators' belief that better decisions about the use of health care could improve the public's health and reduce the cost of care. The Committee on Comparative Effectiveness Research Prioritization was successful in preparing a list 100 top priority CER topics and 10 recommendations for best practices in the field.

Healthcare providers, consumers, researchers and policy makers are inundated with unmanageable amounts of information, including evidence from healthcare research. It has become impossible for all to have the time and resources to find, appraise and interpret this evidence and incorporate it into healthcare decisions. Cochrane Reviews respond to this challenge by identifying, appraising and synthesizing research-based evidence and presenting it in a standardized format, published in The Cochrane Library (www.thecochranelibrary.com). The Cochrane Handbook for Systematic Reviews of Interventions contains methodological guidance for the preparation and maintenance of Cochrane intervention reviews. Written in a clear and accessible format, it is the essential manual for all those preparing, maintaining and reading Cochrane reviews. Many of the principles and methods described here are appropriate for systematic reviews applied to other types of research and to systematic reviews of interventions undertaken by others. It is hoped therefore that this book will be invaluable to all those who want to understand the role of systematic reviews, critically appraise published reviews or perform reviews themselves.

In this open access edited volume, international researchers of the field describe and discuss the systematic review method in its application to research in education. Alongside fundamental methodical considerations, reflections and practice examples are included and provide an introduction and overview on systematic reviews in education research.

There is currently heightened interest in optimizing health care through the generation of new knowledge on the effectiveness of health care services. The United States must substantially strengthen its capacity for assessing evidence on what is known and not known about "what works" in health care. Even the most sophisticated clinicians and consumers struggle to learn which care is appropriate and under what circumstances. Knowing What Works in Health Care looks at the three fundamental health care issues in the United States-setting priorities for evidence assessment, assessing evidence (systematic review), and developing evidence-based clinical practice guidelines-and how each of these contributes to the end goal of effective, practical health care systems. This book provides an overall vision and roadmap for improving how the nation uses scientific evidence to identify the most effective clinical services. Knowing What Works in Health Care gives private and public sector firms, consumers, health care professionals, benefit administrators, and others the authoritative, independent information required for making essential informed health care decisions.