A Theory of Learning and Generalization provides a formal mathematical theory for addressing intuitive questions of the type: How does a machine learn a new concept on the basis of examples? How can a neural network, after sufficient training, correctly predict the output of a previously unseen input? How much training is required to achieve a specified level of accuracy in the prediction? How can one "identify" the dynamical behaviour of a nonlinear control system by observing its input-output behaviour over a finite interval of time? This is the first book to treat the problem of machine learning in conjunction with the theory of empirical processes, the latter being a well-established branch of probability theory. The treatment of both topics side by side leads to new insights, as well as new results in both topics. An extensive references section and open problems will help readers to develop their own work in the field.

How does a machine learn a new concept on the basis of examples? This second edition takes account of important new developments in the field. It also deals extensively with the theory of learning control systems, now comparably mature to learning of neural networks.

This book provides different mathematical frameworks for addressing supervised learning. It is based on a workshop held under the auspices of the Center for Nonlinear Studies at Los Alamos and the Santa Fe Institute in the summer of 1992.

This book covers both classical and modern models in deep learning. The primary focus is on the theory and algorithms of deep learning. The theory and algorithms of neural networks are particularly important for understanding important concepts, so that one can understand the important design concepts of neural architectures in different applications. Why do neural networks work? When do they work better than off-the-shelf machine-learning models? When is depth useful? Why is training neural networks so hard? What are the pitfalls? The book is also rich in discussing different applications in order to give the practitioner a flavor of how neural architectures are designed for different types of problems. Applications associated with many different areas like recommender systems, machine translation, image captioning, image classification, reinforcement-learning based gaming, and text analytics are covered. The chapters of this book span three categories: The basics of neural networks: Many traditional machine learning models can be understood as special cases of neural networks. An emphasis is placed in the first two chapters on understanding the relationship between traditional machine learning and neural networks. Support vector machines, linear/logistic regression, singular value decomposition, matrix factorization, and recommender systems are shown to be special cases of neural networks. These methods are studied together with recent feature engineering methods like word2vec. Fundamentals of neural networks: A detailed discussion of training and regularization is provided in Chapters 3 and 4. Chapters 5 and 6 present radial-basis function (RBF) networks and restricted Boltzmann machines. Advanced topics in neural networks: Chapters 7 and 8 discuss recurrent neural networks and convolutional neural networks. Several advanced topics like deep reinforcement learning, neural Turing machines, Kohonen self-organizing maps, and generative adversarial networks are introduced in Chapters 9 and 10. The book is written for graduate students, researchers, and practitioners. Numerous exercises are available along with a solution manual to aid in classroom teaching. Where possible, an application-centric view is highlighted in order to provide an understanding of the practical uses of each class of techniques.

This thesis presents significant advances in the use of neural networks to study the properties of neutrinos. Machine learning tools like neural networks (NN) can be used to identify the particle types or determine their energies in detectors such as those used in the NOvA neutrino experiment, which studies changes in a beam of neutrinos as it propagates approximately 800 km through the earth. NOvA relies heavily on simulations of the physics processes and the detector response; these simulations work well, but do not match the real experiment perfectly. Thus, neural networks trained on simulated datasets must include systematic uncertainties that account for possible imperfections in the simulation. This thesis presents the first application in HEP of adversarial domain generalization to a regression neural network. Applying domain generalization to problems with large systematic variations will reduce the impact of uncertainties while avoiding the risk of falsely constraining the phase space. Reducing the impact of systematic uncertainties makes NOvA analysis more robust, and improves the significance of experimental results.

This book provides a collection of recent research works addressing theoretical issues on improving the learning process and the generalization of GANs as well as state-of-the-art applications of GANs to various domains of real life. Adversarial learning fascinates the attention of machine learning communities across the world in recent years. Generative adversarial networks (GANs), as the main method of adversarial learning, achieve great success and popularity by exploiting a minimax learning concept, in which two networks compete with each other during the learning process. Their key capability is to generate new data and replicate available data distributions, which are needed in many practical applications, particularly in computer vision and signal processing. The book is intended for academics, practitioners, and research students in artificial intelligence looking to stay up to date with the latest advancements on GANs’ theoretical developments and their applications.