How does a machine learn a new concept on the basis of examples? This second edition takes account of important new developments in the field. It also deals extensively with the theory of learning control systems, now comparably mature to learning of neural networks.

Second, we study the landscape properties of neural network loss functions. In specific, we provide a full characterization of the critical points as well as the global minimizers for linear neural networks and show that the corresponding loss functions have no spurious local minimum. We also show that nonlinear neural networks with ReLU activation function do have spurious local minimum. Lastly, we explore the generalization property of the stochastic gradient descent (SGD) algorithm in nonconvex optimization. Under both un-regularized and regularized setting, we establish the corresponding generalization error bounds for SGD in terms of the on-average variance of the stochastic gradients. Such results lead to improved generalization bounds for SGD and can explain the effect of the random labels on the generalization performance in experiments.

This volume grew out of a workshop designed to bring together researchers from different fields and includes contributions from workers in Bayesian analysis, machine learning, neural nets, PAC and VC theory, classical sampling theory statistics and the statistical physics of learning. The contributions present a bird's-eye view of the subject.

This thesis presents significant advances in the use of neural networks to study the properties of neutrinos. Machine learning tools like neural networks (NN) can be used to identify the particle types or determine their energies in detectors such as those used in the NOvA neutrino experiment, which studies changes in a beam of neutrinos as it propagates approximately 800 km through the earth. NOvA relies heavily on simulations of the physics processes and the detector response; these simulations work well, but do not match the real experiment perfectly. Thus, neural networks trained on simulated datasets must include systematic uncertainties that account for possible imperfections in the simulation. This thesis presents the first application in HEP of adversarial domain generalization to a regression neural network. Applying domain generalization to problems with large systematic variations will reduce the impact of uncertainties while avoiding the risk of falsely constraining the phase space. Reducing the impact of systematic uncertainties makes NOvA analysis more robust, and improves the significance of experimental results.

Providing a broad but in-depth introduction to neural network and machine learning in a statistical framework, this book provides a single, comprehensive resource for study and further research. All the major popular neural network models and statistical learning approaches are covered with examples and exercises in every chapter to develop a practical working understanding of the content. Each of the twenty-five chapters includes state-of-the-art descriptions and important research results on the respective topics. The broad coverage includes the multilayer perceptron, the Hopfield network, associative memory models, clustering models and algorithms, the radial basis function network, recurrent neural networks, principal component analysis, nonnegative matrix factorization, independent component analysis, discriminant analysis, support vector machines, kernel methods, reinforcement learning, probabilistic and Bayesian networks, data fusion and ensemble learning, fuzzy sets and logic, neurofuzzy models, hardware implementations, and some machine learning topics. Applications to biometric/bioinformatics and data mining are also included. Focusing on the prominent accomplishments and their practical aspects, academic and technical staff, graduate students and researchers will find that this provides a solid foundation and encompassing reference for the fields of neural networks, pattern recognition, signal processing, machine learning, computational intelligence, and data mining.