This book provides a practical introduction to the theory and practice of coding and information theory for application in the field of electronic communications. It is written at an introductory level and assumes no prior background in coding or information theory. While the mathematical level is detailed, it is still introductory. Through a discussion that balances theory and practical applications and abandons the traditional "theorem-proof" format, this valuable book presents an overview of digital communication systems and the concept of information. It is written in a easy-to-follow conversational style that integrates practical engineering issues through formal and conceptual discussions of mathematical issues. It also makes extensive use of explicit examples that illustrate methods and theory throughout the book. For the professional, it provides an essential hands-on head start for real-world projects and situations. An essential reference for professional engineers in the field of electronic communications.

This book is an evolution from my book A First Course in Information Theory published in 2002 when network coding was still at its infancy. The last few years have witnessed the rapid development of network coding into a research ?eld of its own in information science. With its root in infor- tion theory, network coding has not only brought about a paradigm shift in network communications at large, but also had signi?cant in?uence on such speci?c research ?elds as coding theory, networking, switching, wireless c- munications,distributeddatastorage,cryptography,andoptimizationtheory. While new applications of network coding keep emerging, the fundamental - sults that lay the foundation of the subject are more or less mature. One of the main goals of this book therefore is to present these results in a unifying and coherent manner. While the previous book focused only on information theory for discrete random variables, the current book contains two new chapters on information theory for continuous random variables, namely the chapter on di?erential entropy and the chapter on continuous-valued channels. With these topics included, the book becomes more comprehensive and is more suitable to be used as a textbook for a course in an electrical engineering department.

This book is based on lectures given by the author at the IBM European Systems Research Institute (ESRI) in Geneva. Information Theory on the syntactic level, as introduced by Claude Shannon in 1949, has many limitations when applied to information processing by computers. But in spite of some obvious shortcomings, the underlyin~ principles are of fundamental importance for systems engineers in understanding the nature of the problems of handling information, its acquisition, storage, processing, and interpretation. The lectures, as presented in this book, attempt to give an exposition of the lovical foundation and basic principles, and to provide at the same time a basis for further study in more specific areas of this expan1in~ theory, such as coding, detection, pattern recognition, and filtering. Most of the problems in Appendix C are intended as extensions of the text, while calling for actjve participation by the stu1ent. Some other problems are direct applications of the theory to specific situations. Some problems require extensive numerical calculations. It is assumed in those cases that the student has access to a computer and that he is capable of writing the necessary programs. The stu1ent is assumed to have a good command of the calculus, and of the theory of probability as well as statistics. Therefore no basic mathematical concepts are discussed in this IV book. The Fourier transform and some related mathematical concepts are introduced in Appendix A.

This book is an introduction to information and coding theory at the graduate or advanced undergraduate level. It assumes a basic knowledge of probability and modern algebra, but is otherwise self- contained. The intent is to describe as clearly as possible the fundamental issues involved in these subjects, rather than covering all aspects in an encyclopedic fashion. The first quarter of the book is devoted to information theory, including a proof of Shannon's famous Noisy Coding Theorem. The remainder of the book is devoted to coding theory and is independent of the information theory portion of the book. After a brief discussion of general families of codes, the author discusses linear codes (including the Hamming, Golary, the Reed-Muller codes), finite fields, and cyclic codes (including the BCH, Reed-Solomon, Justesen, Goppa, and Quadratic Residue codes). An appendix reviews relevant topics from modern algebra.

This fundamental monograph introduces both the probabilistic and algebraic aspects of information theory and coding. It has evolved from the authors' years of experience teaching at the undergraduate level, including several Cambridge Maths Tripos courses. The book provides relevant background material, a wide range of worked examples and clear solutions to problems from real exam papers. It is a valuable teaching aid for undergraduate and graduate students, or for researchers and engineers who want to grasp the basic principles.

Information Theory: Coding Theorems for Discrete Memoryless Systems presents mathematical models that involve independent random variables with finite range. This three-chapter text specifically describes the characteristic phenomena of information theory. Chapter 1 deals with information measures in simple coding problems, with emphasis on some formal properties of Shannon’s information and the non-block source coding. Chapter 2 describes the properties and practical aspects of the two-terminal systems. This chapter also examines the noisy channel coding problem, the computation of channel capacity, and the arbitrarily varying channels. Chapter 3 looks into the theory and practicality of multi-terminal systems. This book is intended primarily for graduate students and research workers in mathematics, electrical engineering, and computer science.