The sixth volume of Rudolf Ahlswede's lectures on Information Theory is focused on Identification Theory. In contrast to Shannon's classical coding scheme for the transmission of a message over a noisy channel, in the theory of identification the decoder is not really interested in what the received message is, but only in deciding whether a message, which is of special interest to him, has been sent or not. There are also algorithmic problems where it is not necessary to calculate the solution, but only to check whether a certain given answer is correct. Depending on the problem, this answer might be much easier to give than finding the solution. ``Easier'' in this context means using fewer resources like channel usage, computing time or storage space. Ahlswede and Dueck's main result was that, in contrast to transmission problems, where the possible code sizes grow exponentially fast with block length, the size of identification codes will grow doubly exponentially fast. The theory of identification has now developed into a sophisticated mathematical discipline with many branches and facets, forming part of the Post Shannon theory in which Ahlswede was one of the leading experts. New discoveries in this theory are motivated both by concrete engineering problems and by explorations of the inherent properties of the mathematical structures. Rudolf Ahlswede wrote: It seems that the whole body of present day Information Theory will undergo serious revisions and some dramatic expansions. In this book we will open several directions of future research and start the mathematical description of communication models in great generality. For some specific problems we provide solutions or ideas for their solutions. The lectures presented in this work, which consists of 10 volumes, are suitable for graduate students in Mathematics, and also for those working in Theoretical Computer Science, Physics, and Electrical Engineering with a background in basic Mathematics. The lectures can be used as the basis for courses or to supplement courses in many ways. Ph.D. students will also find research problems, often with conjectures, that offer potential subjects for a thesis. More advanced researchers may find questions which form the basis of entire research programs. The book also contains an afterword by Gunter Dueck.

This fully updated new edition of a uniquely accessible textbook/reference provides a general introduction to probabilistic graphical models (PGMs) from an engineering perspective. It features new material on partially observable Markov decision processes, causal graphical models, causal discovery and deep learning, as well as an even greater number of exercises; it also incorporates a software library for several graphical models in Python. The book covers the fundamentals for each of the main classes of PGMs, including representation, inference and learning principles, and reviews real-world applications for each type of model. These applications are drawn from a broad range of disciplines, highlighting the many uses of Bayesian classifiers, hidden Markov models, Bayesian networks, dynamic and temporal Bayesian networks, Markov random fields, influence diagrams, and Markov decision processes. Topics and features: Presents a unified framework encompassing all of the main classes of PGMs Explores the fundamental aspects of representation, inference and learning for each technique Examines new material on partially observable Markov decision processes, and graphical models Includes a new chapter introducing deep neural networks and their relation with probabilistic graphical models Covers multidimensional Bayesian classifiers, relational graphical models, and causal models Provides substantial chapter-ending exercises, suggestions for further reading, and ideas for research or programming projects Describes classifiers such as Gaussian Naive Bayes, Circular Chain Classifiers, and Hierarchical Classifiers with Bayesian Networks Outlines the practical application of the different techniques Suggests possible course outlines for instructors This classroom-tested work is suitable as a textbook for an advanced undergraduate or a graduate course in probabilistic graphical models for students of computer science, engineering, and physics. Professionals wishing to apply probabilistic graphical models in their own field, or interested in the basis of these techniques, will also find the book to be an invaluable reference. Dr. Luis Enrique Sucar is a Senior Research Scientist at the National Institute for Astrophysics, Optics and Electronics (INAOE), Puebla, Mexico. He received the National Science Prize en 2016.

Handbook of Probabilistic Models carefully examines the application of advanced probabilistic models in conventional engineering fields. In this comprehensive handbook, practitioners, researchers and scientists will find detailed explanations of technical concepts, applications of the proposed methods, and the respective scientific approaches needed to solve the problem. This book provides an interdisciplinary approach that creates advanced probabilistic models for engineering fields, ranging from conventional fields of mechanical engineering and civil engineering, to electronics, electrical, earth sciences, climate, agriculture, water resource, mathematical sciences and computer sciences. Specific topics covered include minimax probability machine regression, stochastic finite element method, relevance vector machine, logistic regression, Monte Carlo simulations, random matrix, Gaussian process regression, Kalman filter, stochastic optimization, maximum likelihood, Bayesian inference, Bayesian update, kriging, copula-statistical models, and more. - Explains the application of advanced probabilistic models encompassing multidisciplinary research - Applies probabilistic modeling to emerging areas in engineering - Provides an interdisciplinary approach to probabilistic models and their applications, thus solving a wide range of practical problems

A general framework for constructing and using probabilistic models of complex systems that would enable a computer to use available information for making decisions. Most tasks require a person or an automated system to reason—to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.

A self-contained and coherent account of probabilistic techniques, covering: distance measures, kernel rules, nearest neighbour rules, Vapnik-Chervonenkis theory, parametric classification, and feature extraction. Each chapter concludes with problems and exercises to further the readers understanding. Both research workers and graduate students will benefit from this wide-ranging and up-to-date account of a fast- moving field.

Probabilistic models are becoming increasingly important in analysing the huge amount of data being produced by large-scale DNA-sequencing efforts such as the Human Genome Project. For example, hidden Markov models are used for analysing biological sequences, linguistic-grammar-based probabilistic models for identifying RNA secondary structure, and probabilistic evolutionary models for inferring phylogenies of sequences from different organisms. This book gives a unified, up-to-date and self-contained account, with a Bayesian slant, of such methods, and more generally to probabilistic methods of sequence analysis. Written by an interdisciplinary team of authors, it aims to be accessible to molecular biologists, computer scientists, and mathematicians with no formal knowledge of the other fields, and at the same time present the state-of-the-art in this new and highly important field.