This User’s Guide is a resource for investigators and stakeholders who develop and review observational comparative effectiveness research protocols. It explains how to (1) identify key considerations and best practices for research design; (2) build a protocol based on these standards and best practices; and (3) judge the adequacy and completeness of a protocol. Eleven chapters cover all aspects of research design, including: developing study objectives, defining and refining study questions, addressing the heterogeneity of treatment effect, characterizing exposure, selecting a comparator, defining and measuring outcomes, and identifying optimal data sources. Checklists of guidance and key considerations for protocols are provided at the end of each chapter. The User’s Guide was created by researchers affiliated with AHRQ’s Effective Health Care Program, particularly those who participated in AHRQ’s DEcIDE (Developing Evidence to Inform Decisions About Effectiveness) program. Chapters were subject to multiple internal and external independent reviews. More more information, please consult the Agency website: www.effectivehealthcare.ahrq.gov)

Observational studies provide a rich source of data for evaluating causal relationships. Appropriate statistical methods for causal inference should be developed to account for the non-randomized nature of observational studies. Matching design is commonly used to deal with this non-randomized issue as it is robust to the model misspecification. To goal of this work is to use the matching design to perform causal inference in population and subpopulation. Propensity score is a powerful tool for adjusting observed confounding bias when there are a large number of confounders. Relatively few studies have focused on whether the post-matching analysis should adjust for the matching structure when estimate the population treatment effect. In the first part of the thesis, we compare results under different strategies with and without the matching design for both continuous outcome and binary outcome and discuss whether the post-matching should take into account when the treatment effect is homogeneous. \cite{zhang2020accounting} However, treatment effects are likely to be different across different subpopulations, especially in a real-world problem. We then propose a non-parametric matching tree (MT) to tackle both confounding adjustment and subgroup identification at the same time by combining the machine learning methods with matching designs. We prove that it produces unbiased subpopulation treatment effect estimators. To evaluate the performance of the proposed method, we run extensive simulation studies to compare it with popular tree-based causal inference methods. We apply the proposed method to examine the impact of Tobramycin for the patients' first pseudomonas aeruginosa chronic infection in Cystic Fibrosis disease in the U.S. We finally discuss limitations and potential future works.

Healthcare providers, consumers, researchers and policy makers are inundated with unmanageable amounts of information, including evidence from healthcare research. It has become impossible for all to have the time and resources to find, appraise and interpret this evidence and incorporate it into healthcare decisions. Cochrane Reviews respond to this challenge by identifying, appraising and synthesizing research-based evidence and presenting it in a standardized format, published in The Cochrane Library (www.thecochranelibrary.com). The Cochrane Handbook for Systematic Reviews of Interventions contains methodological guidance for the preparation and maintenance of Cochrane intervention reviews. Written in a clear and accessible format, it is the essential manual for all those preparing, maintaining and reading Cochrane reviews. Many of the principles and methods described here are appropriate for systematic reviews applied to other types of research and to systematic reviews of interventions undertaken by others. It is hoped therefore that this book will be invaluable to all those who want to understand the role of systematic reviews, critically appraise published reviews or perform reviews themselves.

The goal of this study is to better understand how methods for estimating treatment effects of latent groups operate. In particular, the authors identify where violations of assumptions can lead to biased estimates, and explore how covariates can be critical in the estimation process. For each set of approaches, the authors first review the assumptions necessary for identification and discuss practical issues that arise in estimation; second, they then examine how covariates allow for improved estimation, and determine the conditions necessary for using covariates to identify causal effects in latent groups; and third, they then compare the different methods using simulation studies built from datasets constructed by imputing missing class membership and potential outcomes from real-world studies. This allows for examining the performance of the different techniques under a variety of plausible circumstances. Analyzed is data from the Job Search Intervention Study (JOBS II), a randomized evaluation of an intervention for unemployed workers consisting of a series of training sessions and also the Head Start Impact Study, a large-scale randomized evaluation of the Head Start program in which children randomized to treatment were offered a seat in a classroom in a Head Start program. The authors conclude that, in practice, randomized trials should attempt to collect such covariates by, for example, having expert assessment of likelihood of compliance collected at baseline and that for identification, many methods require assumptions that are quite strong.

This paper proposes strategies for defining, identifying, and estimating features of treatment-effect distributions in contexts where multiple outcomes are of interest. After describing existing empirical approaches used in such settings, the paper develops a notion of treatment preference that is shown to be a feature of standard treatment-effect analysis in the single-outcome case. Focusing largely on binary outcomes, treatment-preference probability treatment effects (PTEs) are defined and are seen to correspond to familiar average treatment effects in the single-outcome case. The paper suggests seven possible characterizations of treatment preference appropriate to multiple-outcome contexts. Under standard assumptions about unconfoundedness of treatment assignment, the PTEs are shown to be point identified for three of the seven characterizations and set identified for the other four. Probability bounds are derived and empirical approaches to estimating the bounds--or the PTEs themselves in the point-identified cases--are suggested. These empirical approaches are straightforward, involving in most instances little more than estimation of binary-outcome probability models of what are commonly known as composite outcomes. The results are illustrated with simulated data and in analyses of two microdata samples. Finally, the main results are extended to situations where the component outcomes are ordered or categorical.