This book provides an up-to-date introduction to information theory. In addition to the classical topics discussed, it provides the first comprehensive treatment of the theory of I-Measure, network coding theory, Shannon and non-Shannon type information inequalities, and a relation between entropy and group theory. ITIP, a software package for proving information inequalities, is also included. With a large number of examples, illustrations, and original problems, this book is excellent as a textbook or reference book for a senior or graduate level course on the subject, as well as a reference for researchers in related fields.

From the reviews: "Bioinformaticians are facing the challenge of how to handle immense amounts of raw data, [...] and render them accessible to scientists working on a wide variety of problems. [This book] can be such a tool." IEEE Engineering in Medicine and Biology

An important text that offers an in-depth guide to how information theory sets the boundaries for data communication In an accessible and practical style, Information and Communication Theory explores the topic of information theory and includes concrete tools that are appropriate for real-life communication systems. The text investigates the connection between theoretical and practical applications through a wide-variety of topics including an introduction to the basics of probability theory, information, (lossless) source coding, typical sequences as a central concept, channel coding, continuous random variables, Gaussian channels, discrete input continuous channels, and a brief look at rate distortion theory. The author explains the fundamental theory together with typical compression algorithms and how they are used in reality. He moves on to review source coding and how much a source can be compressed, and also explains algorithms such as the LZ family with applications to e.g. zip or png. In addition to exploring the channel coding theorem, the book includes illustrative examples of codes. This comprehensive text: Provides an adaptive version of Huffman coding that estimates source distribution Contains a series of problems that enhance an understanding of information presented in the text Covers a variety of topics including optimal source coding, channel coding, modulation and much more Includes appendices that explore probability distributions and the sampling theorem Written for graduate and undergraduate students studying information theory, as well as professional engineers, master’s students, Information and Communication Theory offers an introduction to how information theory sets the boundaries for data communication.

Entropy, mutual information and divergence measure the randomness, dependence and dissimilarity, respectively, of random objects. In addition to their prominent role in information theory, they have found numerous applications, among others, in probability theory statistics, physics, chemistry, molecular biology, ecology, bioinformatics, neuroscience, machine learning, linguistics, and finance. Many of these applications require a universal estimate of information measures which does not assume knowledge of the statistical properties of the observed data. Over the past few decades, several nonparametric algorithms have been proposed to estimate information measures. Universal Estimation of Information Measures for Analog Sources presents a comprehensive survey of universal estimation of information measures for memoryless analog (real-valued or real vector-valued) sources with an emphasis on the estimation of mutual information and divergence and their applications. The book reviews the consistency of the universal algorithms and the corresponding sufficient conditions as well as their speed of convergence. Universal Estimation of Information Measures for Analog Sources provides a comprehensive review of an increasingly important topic in Information Theory. It will be of interest to students, practitioners and researchers working in Information Theory

"This book is highly recommended for all those whose interests lie in the fields that deal with any kind of information measures. It will also find readers in the field of functional analysis..".Mathematical Reviews

The book deals with the application of various measures of information like the entropy, divergence, inaccuracy, etc. in modelling lifetimes of devices or equipment in reliability analysis. This is an emerging area of study and research during the last two decades and is of potential interest in many fields. In this work the classical measures of uncertainty are sufficiently modified to meet the needs of lifetime data analysis. The book provides an exhaustive collection of materials in a single volume to make it a comprehensive source of reference. The first treatise on the subject. It brings together the work that have appeared in journals on different disciplines. It will serve as a text for graduate students and practioners of special studies in information theory, as well as statistics and as a reference book for researchers. The book contains illustrative examples, tables and figures for clarifying the concepts and methodologies, the book is self-contained. It helps students to access information relevant to careers in industry, engineering, applied statistics, etc.

This book develops applications of novel generalizations of fuzzy information measures in the field of pattern recognition, medical diagnosis, multi-criteria and multi-attribute decision making and suitability in linguistic variables. The focus of this presentation lies on introducing consistently strong and efficient generalizations of information and information-theoretic divergence measures in fuzzy and intuitionistic fuzzy environment covering different practical examples. The target audience comprises primarily researchers and practitioners in the involved fields but the book may also be beneficial for graduate students.