Multiple comparison tests are quite prevalent in clinical trials to increase efficiency and the chance of finding treatment or drug effects. However, making multiple comparisons will lead to the potential inflation of the Type I error rate. How well one test procedure controls the rate of false-positive conclusions becomes the main concern nowadays. A novel method named Covering Principle has been proposed recently. It provides a new approach to designing multiple testing procedures from the angle of rejection regions in the sample space, rather than in the parameter space, which uses in the methods based on the closed testing and partitioning principle. This paper presented several cases studies and a simulation study using Holm's, Fixed and Fallback based on the Covering Principle. Meanwhile, the power comparison results between the Covering Principle and the graphical method are exhibited and analyzed in this paper.

Useful Statistical Approaches for Addressing Multiplicity IssuesIncludes practical examples from recent trials Bringing together leading statisticians, scientists, and clinicians from the pharmaceutical industry, academia, and regulatory agencies, Multiple Testing Problems in Pharmaceutical Statistics explores the rapidly growing area of multiple c

Statistical Power Analysis is a nontechnical guide to power analysis in research planning that provides users of applied statistics with the tools they need for more effective analysis. The Second Edition includes: * a chapter covering power analysis in set correlation and multivariate methods; * a chapter considering effect size, psychometric reliability, and the efficacy of "qualifying" dependent variables and; * expanded power and sample size tables for multiple regression/correlation.

In the context of large-scale multiple testing, hypotheses are often accompanied with certain prior information. In chapter 2, we present a single-index modulated multiple testing procedure, which maintains control of the false discovery rate while incorporating prior information, by assuming the availability of a bivariate p-value for each hypothesis. To find the optimal rejection region for the bivariate p-value, we propose a criteria based on the ratio of probability density functions of the bivariate p-value under the true null and non-null. This criteria in the bivariate normal setting further motivates us to project the bivariate p-value to a single index p-value, for a wide range of directions. The true null distribution of the single index p-value is estimated via parametric and nonparametric approaches, leading to two procedures for estimating and controlling the false discovery rate. To derive the optimal projection direction, we propose a new approach based on power comparison, which is further shown to be consistent under some mild conditions. Multiple testing based on chi-squared test statistics is commonly used in many scientific fields such as genomics research and brain imaging studies. However, the challenges associated with designing a formal testing procedure when there exists a general dependence structure across the chi-squared test statistics have not been well addressed. In chapter 3, we propose a Factor Connected procedure to fill in this gap. We first adopt a latent factor structure to construct a testing framework for approximating the false discovery proportion (FDP) for a large number of highly correlated chi-squared test statistics with finite degrees of freedom k. The testing framework is then connected to simultaneously testing k linear constraints in a large dimensional linear factor model involved with some observable and unobservable common factors, resulting in a consistent estimator of FDP based on the associated unadjusted p-values.

Adopting a unifying theme based on maximum statistics, Multiple Comparisons Using R describes the common underlying theory of multiple comparison procedures through numerous examples. It also presents a detailed description of available software implementations in R. The R packages and source code for the analyses are available at http://CRAN.R-project.org After giving examples of multiplicity problems, the book covers general concepts and basic multiple comparisons procedures, including the Bonferroni method and Simes’ test. It then shows how to perform parametric multiple comparisons in standard linear models and general parametric models. It also introduces the multcomp package in R, which offers a convenient interface to perform multiple comparisons in a general context. Following this theoretical framework, the book explores applications involving the Dunnett test, Tukey’s all pairwise comparisons, and general multiple contrast tests for standard regression models, mixed-effects models, and parametric survival models. The last chapter reviews other multiple comparison procedures, such as resampling-based procedures, methods for group sequential or adaptive designs, and the combination of multiple comparison procedures with modeling techniques. Controlling multiplicity in experiments ensures better decision making and safeguards against false claims. A self-contained introduction to multiple comparison procedures, this book offers strategies for constructing the procedures and illustrates the framework for multiple hypotheses testing in general parametric models. It is suitable for readers with R experience but limited knowledge of multiple comparison procedures and vice versa. See Dr. Bretz discuss the book.

Multiple testing, a situation where multiple hypothesis tests are performed simultaneously, is a core research topic in statistics that arises in almost every scientific field. When more hypotheses are tested, more errors are bound to occur. Controlling the false discovery rate (FDR) [BH95], which is the expected proportion of falsely rejected null hypotheses among all rejections, is an important challenge for making meaningful inferences. Throughout the dissertation, we analyze the asymptotic performance of several FDR-controlling procedures under different multiple testing settings. In Chapter 1, we study the famous Benjamini-Hochberg (BH) method [BH95] which often serves as benchmark among FDR-controlling procedures, and show that it is asymptotic optimal in a stylized setting. We then prove that a distribution-free FDR control method of Barber and Candès [FBC15], which only requires the (unknown) null distribution to be symmetric, can achieve the same asymptotic performance as the BH method, thus is also optimal. Chapter 2 proposes an interval-type procedure which identifies the longest interval with the estimated FDR under a given level and rejects the corresponding hypotheses with P-values lying inside the interval. Unlike the threshold approaches, this procedure scans over all intervals with the left point not necessary being zero. We show that this scan procedure provides strong control of the asymptotic false discovery rate. In addition, we investigate its asymptotic false non-discovery rate (FNR), deriving conditions under which it outperforms the BH procedure. In Chapter 3, we consider an online multiple testing problem where the hypotheses arrive sequentially in a stream, and investigate two procedures proposed by Javanmard and Montanari [JM15] which control FDR in an online manner. We quantify their asymptotic performance in the same location models as in Chapter 1 and compare their power with the (static) BH method. In Chapter 4, we propose a new class of powerful online testing procedures which incorporates the available contextual information, and prove that any rule in this class controls the online FDR under some standard assumptions. We also derive a practical algorithm that can make more empirical discoveries in an online fashion, compared to the state-of-the-art procedures.

Combines recent developments in resampling technology (including the bootstrap) with new methods for multiple testing that are easy to use, convenient to report and widely applicable. Software from SAS Institute is available to execute many of the methods and programming is straightforward for other applications. Explains how to summarize results using adjusted p-values which do not necessitate cumbersome table look-ups. Demonstrates how to incorporate logical constraints among hypotheses, further improving power.