High-dimensional linear models play an important role in the analysis of modern data sets. Although the estimation problem has been well understood, there is still a paucity of methods and theories on the inference problem for high-dimensional linear models. This thesis focuses on statistical inference for high-dimensional linear models and consists of the following three parts. 1. The first part of the thesis considers confidence intervals for linear functionals in high-dimensional linear regression. We first establish the convergence rates of the minimax expected length for confidence intervals. Furthermore, we investigate the problem of adaptation to sparsity for the construction of confidence intervals and identify the regimes in which it is possible to construct adaptive confidence intervals. 2. In the second part of the thesis, we consider point and interval estimation of the lq loss of a given estimator in high-dimensional linear regression. For the class of rate-optimal estimators, we establish the minimax rates for estimating their lq losses, the minimax expected length of confidence intervals for their lq losses and the possibility of adaptivity of confidence intervals for their lq losses. 3. In the third part of the thesis, we consider the problem in the framework of high-dimensional instrumental variable regression and construct confidence intervals for the treatment effect in the presence of possibly invalid instrumental variables. We develop a novel selection procedure, Two-Stage Hard Thresholding (TSHT) to select valid instrumental variables and construct honest confidence intervals for the treatment effect using the selected instrumental variables.

Variable selection procedures for high dimensional data have been proposed and studied by a large amount of literature in the last few years. Most of the previous research focuses on the selection properties as well as the point estimation properties. In this paper, our goal is to construct the confidence intervals for some low-dimensional parameters in the high-dimensional setting. The models we study are the partially penalized linear and accelerated failure time models in the high-dimensional setting. In our model setup, all variables are split into two groups. The first group consists of a relatively small number of variables that are more interesting. The second group consists of a large amount of variables that can be potentially correlated with the response variable. We propose an approach that selects the variables from the second group and produces confidence intervals for the parameters in the first group. We show the sign consistency of the selection procedure and give a bound on the estimation error. Based on this result, we provide the sufficient conditions for the asymptotic normality of the low-dimensional parameters. The high-dimensional selection consistency and the low-dimensional asymptotic normality are developed for both linear and AFT models with high-dimensional data.

Statistical inference under high dimensional modelings has attracted much attention due to its wide applications in many fields. In this dissertation, I propose new methods for statistical inference in high dimensional models from three aspects: inference in high dimensional semiparametric models, inference in high dimensional matrix-valued data, and inference in high dimensional measurement error misspecified models. The first project studied statistical inference in high dimensional partially linear single index models. Firstly a profile partial penalized least squares estimator for parameter estimates for the model is proposed, and its asymptotic properties are given. Then an F-type test statistic for testing the parametric components is proposed, and its theoretical properties are established. I then propose a new test for the specification testing problem of the nonparametric components. Finally, simulation studies and empirical analysis of a real-world data set are conducted to illustrate the performance of the proposed testing procedure. The second project proposes new testing procedures in high dimensional matrix-valued data. Rank is an essential attribute for a matrix. A new type of statistic is proposed, which can make inferences on the rank of the matrix-valued data. I firstly give the theoretical property of its oracle version. To overcome the problem of empirical error accumulation, a new type of sparse SVD method is proposed, and its theoretical properties are given. Based on the newly proposed sparse SVD method, I provide a sample version statistic. Theoretical properties of this sample version statistic are given. Simulation studies and two applications to surveillance video data are provided to illustrate the performance of our newly proposed method. The third project proposes a new testing method in misspecified measurement error models. The testing method can work when there is potential model misspecification and measurement error in the model. Firstly its property is studied under the low dimensional setting. Then I develop it to the high dimensional setting. Further, I propose a method that can be adaptive to the sparsity level of the true parameters under the high dimensional setting. Simulation studies and one application to a clinical trial data set are given.

Modern statistics deals with large and complex data sets, and consequently with models containing a large number of parameters. This book presents a detailed account of recently developed approaches, including the Lasso and versions of it for various models, boosting methods, undirected graphical modeling, and procedures controlling false positive selections. A special characteristic of the book is that it contains comprehensive mathematical theory on high-dimensional statistics combined with methodology, algorithms and illustrations with real data examples. This in-depth approach highlights the methods’ great potential and practical applicability in a variety of settings. As such, it is a valuable resource for researchers, graduate students and experts in statistics, applied mathematics and computer science.

This dissertation is dedicated to studying the problem of constructing asymptotically valid confidence intervals for change points in high-dimensional linear models, where the number of parameters may vastly exceed the sampling period.In Chapter 2, we develop an algorithmic estimator for a single change point and establish the optimal rate of estimation, Op(Îl 8́22 ), where Îl represents the jump size under a high dimensional scaling. The optimal result ensures the existence of limiting distributions. Asymptotic distributions are derived under both vanishing and non-vanishing regimes of jump size. In the former case, it corresponds to the argmax of a two-sided Brownian motion, while in the latter case to the argmax of a two-sided random walk, both with negative drifts. We also provide the relationship between the two distributions, which allows construction of regime (vanishing vs non-vanishing) adaptive confidence intervals.In Chapter 3, we extend our analysis to the statistical inference for multiple change points in high-dimensional linear regression models. We develop locally refitted estimators and evaluate their convergence rates both component-wise and simultaneously. Following similar manner as in Chapter 2, we achieve an optimal rate of estimation under the component-wise scenario, which guarantees the existence of limiting distributions. While we also establish the simultaneous rate which is the sharpest available by a logarithmic factor. Component-wise and joint limiting distributions are derived under vanishing and non-vanishing regimes of jump sizes, demonstrating the relationship between distributions in the two regimes.Lastly in Chapter 4, we introduce a novel implementation method for finding preliminary change points estimates via integer linear programming, which has not yet been explored in the current literature.Overall, this dissertation provides a comprehensive framework for inference on single and multiple change points in high-dimensional linear models, offering novel and efficient algorithms with strong theoretical guarantees. All theoretical results are supported by Monte Carlo simulations.

Regression models are very common for statistical inference, especially linear regression models with Gaussian noise. But in many modern scientific applications with large-scale datasets, the number of samples is small relative to the number of model parameters, which is the so-called high- dimensional setting. Directly applying classical linear regression models to high-dimensional data is ill-posed. Thus it is necessary to impose additional assumptions for regression coefficients to make high-dimensional statistical analysis possible. Regularization methods with sparsity assumptions have received substantial attention over the past two decades. But there are still some open questions regarding high-dimensional statistical analysis. Firstly, most literature provides statistical analysis for high-dimensional linear models with Gaussian noise, it is unclear whether similar results still hold if we are no longer in the Gaussian setting. To answer this question under Poisson setting, we study the minimax rates and provide an implementable convex algorithm for high-dimensional Poisson inverse problems under weak sparsity assumption and physical constraints. Secondly, much of the theory and methodology for high-dimensional linear regression models are based on the assumption that independent variables are independent of each other or have weak correlations. But it is possible that this assumption is not satisfied that some features are highly correlated with each other. It is natural to ask whether it is still possible to make high-dimensional statistical inference with high-correlated designs. Thus we provide a graph-based regularization method for high-dimensional regression models with high-correlated designs along with theoretical guarantees.

In the last ten years, there has been increasing interest and activity in the general area of partially linear regression smoothing in statistics. Many methods and techniques have been proposed and studied. This monograph hopes to bring an up-to-date presentation of the state of the art of partially linear regression techniques. The emphasis is on methodologies rather than on the theory, with a particular focus on applications of partially linear regression techniques to various statistical problems. These problems include least squares regression, asymptotically efficient estimation, bootstrap resampling, censored data analysis, linear measurement error models, nonlinear measurement models, nonlinear and nonparametric time series models.