This book covers the most recent developments in adaptive dynamic programming (ADP). The text begins with a thorough background review of ADP making sure that readers are sufficiently familiar with the fundamentals. In the core of the book, the authors address first discrete- and then continuous-time systems. Coverage of discrete-time systems starts with a more general form of value iteration to demonstrate its convergence, optimality, and stability with complete and thorough theoretical analysis. A more realistic form of value iteration is studied where value function approximations are assumed to have finite errors. Adaptive Dynamic Programming also details another avenue of the ADP approach: policy iteration. Both basic and generalized forms of policy-iteration-based ADP are studied with complete and thorough theoretical analysis in terms of convergence, optimality, stability, and error bounds. Among continuous-time systems, the control of affine and nonaffine nonlinear systems is studied using the ADP approach which is then extended to other branches of control theory including decentralized control, robust and guaranteed cost control, and game theory. In the last part of the book the real-world significance of ADP theory is presented, focusing on three application examples developed from the authors’ work: • renewable energy scheduling for smart power grids;• coal gasification processes; and• water–gas shift reactions. Researchers studying intelligent control methods and practitioners looking to apply them in the chemical-process and power-supply industries will find much to interest them in this thorough treatment of an advanced approach to control.

There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods: • infinite-horizon control for which the difficulty of solving partial differential Hamilton–Jacobi–Bellman equations directly is overcome, and proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences; • finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control; • nonlinear games for which a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point. Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium. In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time: • establishes the fundamental theory involved clearly with each chapter devoted to a clearly identifiable control paradigm; • demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and • shows how ADP methods can be put to use both in simulation and in real applications. This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.

This book represents the third of three volumes offering a complete reinterpretation and restructuring of Keynesian macroeconomics and a detailed investigation of the disequilibrium adjustment processes characterizing the financial, the goods and the labour markets and their interaction. This book offers a full treatment of the interlinkages between the real and the financial markets, including an analysis of banking, credit, and endogenous money and asset markets. It remains critical of quite frequently used conventional macro models that have dropped the tradition of studying the macroeconomic feedback channels, well-known in the history of macroeconomics. Those feedback mechanisms are known to have the potential for instabilities with respect to real markets, price dynamics and financial markets. In this volume a particular emphasis is given to the financial-real interaction. The research in this book with its focus on Keynesian propagation mechanisms provides a unique alternative to the black-box shock-absorber approaches that dominate modern macroeconomics. The main conclusion of the work is that policy makers need to reconsider Keynesian ideas, but in the modern form in which they are expressed in this volume. Reconstructing Keynesian Macroeconomics will be of interest to students and researchers who want to look at alternatives to the mainstream macrodynamics that emerged from the Monetarist critique of Keynesianism. This book will also engage central bankers and macroeconomic policy makers.

This book presents a class of novel, self-learning, optimal control schemes based on adaptive dynamic programming techniques, which quantitatively obtain the optimal control schemes of the systems. It analyzes the properties identified by the programming methods, including the convergence of the iterative value functions and the stability of the system under iterative control laws, helping to guarantee the effectiveness of the methods developed. When the system model is known, self-learning optimal control is designed on the basis of the system model; when the system model is not known, adaptive dynamic programming is implemented according to the system data, effectively making the performance of the system converge to the optimum. With various real-world examples to complement and substantiate the mathematical analysis, the book is a valuable guide for engineers, researchers, and students in control science and engineering.

This book presents the state of the art in the relatively new field of dynamic economic modelling with regime switches. The contributions, written by prominent scholars in the field, focus on dynamic decision problems with regime changes in underlying dynamics or objectives. Such changes can be externally driven or internally induced by decisions. Utilising the most advanced mathematical methods in optimal control and dynamic game theory, the authors address a broad range of topics, including capital accumulation, innovations, financial decisions, population economics, environmental and resource economics, institutional change and the dynamics of addiction. Given its scope, the book will appeal to all scholars interested in mathematical and quantitative economics.