Since its origins in the 1940s, the subject of decision making under uncertainty has grown into a diversified area with application in several branches of engineering and in those areas of the social sciences concerned with policy analysis and prescription. These approaches required a computing capacity too expensive for the time, until the ability to collect and process huge quantities of data engendered an explosion of work in the area. This book provides succinct and rigorous treatment of the foundations of stochastic control; a unified approach to filtering, estimation, prediction, and stochastic and adaptive control; and the conceptual framework necessary to understand current trends in stochastic control, data mining, machine learning, and robotics.

Optimization of Stochastic Systems

The H∞ control has been one of the important robust control approaches since the 1980s. This book extends the area to nonlinear stochastic H2/H∞ control, and studies more complex and practically useful mixed H2/H∞ controller synthesis rather than the pure H∞ control. Different from the commonly used convex optimization method, this book applies the Nash game approach to give necessary and sufficient conditions for the existence and uniqueness of the mixed H2/H∞ control. Researchers will benefit from our detailed exposition of the stochastic mixed H2/H∞ control theory, while practitioners can apply our efficient algorithms to address their practical problems.

Identifying the input-output relationship of a system or discovering the evolutionary law of a signal on the basis of observation data, and applying the constructed mathematical model to predicting, controlling or extracting other useful information constitute a problem that has been drawing a lot of attention from engineering and gaining more and more importance in econo metrics, biology, environmental science and other related areas. Over the last 30-odd years, research on this problem has rapidly developed in various areas under different terms, such as time series analysis, signal processing and system identification. Since the randomness almost always exists in real systems and in observation data, and since the random process is sometimes used to model the uncertainty in systems, it is reasonable to consider the object as a stochastic system. In some applications identification can be carried out off line, but in other cases this is impossible, for example, when the structure or the parameter of the system depends on the sample, or when the system is time-varying. In these cases we have to identify the system on line and to adjust the control in accordance with the model which is supposed to be approaching the true system during the process of identification. This is why there has been an increasing interest in identification and adaptive control for stochastic systems from both theorists and practitioners.

This unified survey focuses on linear discrete-time systems and explores natural extensions to nonlinear systems. It emphasizes discrete-time systems, summarizing theoretical and practical aspects of a large class of adaptive algorithms. 1984 edition.

The book reviews developments in the following fields: optimal adaptive control; online differential games; reinforcement learning principles; and dynamic feedback control systems.

Stochastic control is a very active area of research. This monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels, that of practice and that of mathematical development. It is broadly accessible for graduate students and researchers.