During the last few years, computers have evolved from pure number crunching machines to "intelligent" problem solving tools. Increasing effort has been spent on the investigation of new approaches and the application of solutions to real world problems. In this way, exciting new techniques have evolved providing support for an increasing number of technical and economical aspects. Applications range from the design and development of ultra highly integrated circuits to totally new man-machine interfaces, from software engineering tools to fault diagnosis systems, from decision support to even the analysis of unemployment. Following a first joint workshop on Advanced Information Processing held in July 1988 at the Institute for Problems of Informatics of the USSR Academy of Sciences (IPIAN) at Moscow, this was the second time that scientists and researchers from the USSR Academy of Sciences and Siemens AG, Corporate Research and Development, exchanged results and discussed recent advances in the field of applied computer sciences. Initiated by Prof. Dr. I. Mizin, Corresponding Member of the USSR Academy of Sciences and Director of IPIAN, and Prof. Dr. H. Schwartzel, Vice President of the Siemens AG and Head of the Applied Computer Science & Software Department, a joint symposium was arranged at the USSR Academy of Sciences in Moscow on June 5th and 6th 1990. The meetings on Information Processing and Software and Systems Design Automation provided a basis both for presentations of ongoing research and for discussions about specific problems.

Information Processing is a key area of research and development and the symposium presented state-of-the-art reports on some of the areas which are of relevance in automatic control: fault diagnosis and system reliability. Papers also covered the role of expert systems and other knowledge based systems, which are needed, to cope with the vast quantities of data generated by large scale systems. This volume should be considered essential reading for anyone involved in this rapidly developing area.

"Info-metrics is a framework for rational inference on the basis of limited, or insufficient, information. It is the science of modeling, reasoning, and drawing inferences under conditions of noisy and insufficient information. Info-metrics has its roots in information theory (Shannon, 1948), Bernoulli's and Laplace's principle of insufficient reason (Bernoulli, 1713) and its offspring the principle of maximum entropy (Jaynes, 1957). It is an interdisciplinary framework situated at the intersection of information theory, statistical inference, and decision-making under uncertainty. Within a constrained optimization setup, info-metrics provides a simple way for modeling and understanding all types of systems and problems. It is a framework for processing the available information with minimal reliance on assumptions and information that cannot be validated. Quite often a model cannot be validated with finite data. Examples include biological, social and behavioral models, as well as models of cognition and knowledge. The info-metrics framework extends naturally for tackling these types of common problems"--

Theory of Neural Information Processing Systems provides an explicit, coherent, and up-to-date account of the modern theory of neural information processing systems. It has been carefully developed for graduate students from any quantitative discipline, including mathematics, computer science, physics, engineering or biology, and has been thoroughly class-tested by the authors over a period of some 8 years. Exercises are presented throughout the text and notes on historical background and further reading guide the student into the literature. All mathematical details are included and appendices provide further background material, including probability theory, linear algebra and stochastic processes, making this textbook accessible to a wide audience.

This book covers the theoretical foundations of advanced mean field methods, explores the relation between the different approaches, examines the quality of the approximation obtained, and demonstrates their application to various areas of probabilistic modeling. A major problem in modern probabilistic modeling is the huge computational complexity involved in typical calculations with multivariate probability distributions when the number of random variables is large. Because exact computations are infeasible in such cases and Monte Carlo sampling techniques may reach their limits, there is a need for methods that allow for efficient approximate computations. One of the simplest approximations is based on the mean field method, which has a long history in statistical physics. The method is widely used, particularly in the growing field of graphical models. Researchers from disciplines such as statistical physics, computer science, and mathematical statistics are studying ways to improve this and related methods and are exploring novel application areas. Leading approaches include the variational approach, which goes beyond factorizable distributions to achieve systematic improvements; the TAP (Thouless-Anderson-Palmer) approach, which incorporates correlations by including effective reaction terms in the mean field theory; and the more general methods of graphical models. Bringing together ideas and techniques from these diverse disciplines, this book covers the theoretical foundations of advanced mean field methods, explores the relation between the different approaches, examines the quality of the approximation obtained, and demonstrates their application to various areas of probabilistic modeling.