Bayesian nonparametric methods have become increasingly popular in machine learning for their ability to allow the data to determine model complexity. In particular, Bayesian nonparametric versions of common latent variable models can learn as effective dimension of the latent space. Examples include mixture models, latent feature models and topic models, where the number of components, features, or topics need not be specified a priori. A drawback of many of these models is that they assume the observations are exchangeable, that is, any dependencies between observations are ignored. This thesis contributes general methods to incorporate covariates into Bayesian nonparametric models and inference algorithms to learn with these models. First, we will present a flexible class of dependent Bayesian nonparametric priors to induce covariate-dependence into a variety of latent variable models used in machine learning. The proposed framework has nice analytic properites and admits a simple inference algorithm. We show how the framework can be used to construct a covariate-dependent latent feature model and a time-varying topic model. Second, we describe the first general purpose inference algorithm for a large family of dependent mixture models. Using the idea of slice-sampling, the algorithm is truncation-free and fast, showing that inference can de done efficiently despite the added complexity that covariate-dependence entails. Last, we construct a Bayesian nonparametric framework to couple multiple latent variable models and apply the framework to learning from multiple views of data.

This book is the first systematic treatment of Bayesian nonparametric methods and the theory behind them. It will also appeal to statisticians in general. The book is primarily aimed at graduate students and can be used as the text for a graduate course in Bayesian non-parametrics.

Bayesian nonparametrics works - theoretically, computationally. The theory provides highly flexible models whose complexity grows appropriately with the amount of data. Computational issues, though challenging, are no longer intractable. All that is needed is an entry point: this intelligent book is the perfect guide to what can seem a forbidding landscape. Tutorial chapters by Ghosal, Lijoi and Prünster, Teh and Jordan, and Dunson advance from theory, to basic models and hierarchical modeling, to applications and implementation, particularly in computer science and biostatistics. These are complemented by companion chapters by the editors and Griffin and Quintana, providing additional models, examining computational issues, identifying future growth areas, and giving links to related topics. This coherent text gives ready access both to underlying principles and to state-of-the-art practice. Specific examples are drawn from information retrieval, NLP, machine vision, computational biology, biostatistics, and bioinformatics.

This book reviews nonparametric Bayesian methods and models that have proven useful in the context of data analysis. Rather than providing an encyclopedic review of probability models, the book’s structure follows a data analysis perspective. As such, the chapters are organized by traditional data analysis problems. In selecting specific nonparametric models, simpler and more traditional models are favored over specialized ones. The discussed methods are illustrated with a wealth of examples, including applications ranging from stylized examples to case studies from recent literature. The book also includes an extensive discussion of computational methods and details on their implementation. R code for many examples is included in online software pages.

Recent progress in Bayesian methods largely focus on non-conjugate models featured with extensive use of black-box functions: continuous functions implemented with neural networks. Using deep neural networks, Bayesian models can reasonably fit big data while at the same time capturing model uncertainty. This thesis targets at a more challenging problem: how do we model general random objects, including discrete ones, using random functions? Our conclusion is: many (discrete) random objects are in nature a composition of Poisson processes and random functions}. Thus, all discreteness is handled through the Poisson process while random functions captures the rest complexities of the object. Thus the title: composing deep learning and Bayesian nonparametric methods. This conclusion is not a conjecture. In spacial cases such as latent feature models , we can prove this claim by working on infinite dimensional spaces, and that is how Bayesian nonparametric kicks in. Moreover, we will assume some regularity assumptions on random objects such as exchangeability.