This book is devoted to a specific problem in the general theory of automatic control -- sequential control under conditions of incomplete information. The main results concern the case in which at each moment of (continuous) time only a finite number of controls are admissible and the results of control action are represented by realizations of random variables whose distributions at a given control correspond to one of several alternative hypotheses. The analysis is conducted in a Bayesian framework.

The second edition of a rigorous and example-driven introduction to topics in economic dynamics that emphasizes techniques for modeling dynamic systems. This text provides an introduction to the modern theory of economic dynamics, with emphasis on mathematical and computational techniques for modeling dynamic systems. Written to be both rigorous and engaging, the book shows how sound understanding of the underlying theory leads to effective algorithms for solving real-world problems. The material makes extensive use of programming examples to illustrate ideas, bringing to life the abstract concepts in the text. Key topics include algorithms and scientific computing, simulation, Markov models, and dynamic programming. Part I introduces fundamentals and part II covers more advanced material. This second edition has been thoroughly updated, drawing on recent research in the field. New for the second edition: “Programming-language agnostic” presentation using pseudocode. New chapter 1 covering conceptual issues concerning Markov chains such as ergodicity and stability. New focus in chapter 2 on algorithms and techniques for program design and high-performance computing. New focus on household problems rather than optimal growth in material on dynamic programming. Solutions to many exercises, code, and other resources available on a supplementary website.

Some recent developments in the mathematics of optimization, including the concepts of invexity and quasimax, have not yet been applied to models of economic growth, and to finance and investment. Their applications to these areas are shown in this book.

Stochastic Modelling of Social Processes provides information pertinent to the development in the field of stochastic modeling and its applications in the social sciences. This book demonstrates that stochastic models can fulfill the goals of explanation and prediction. Organized into nine chapters, this book begins with an overview of stochastic models that fulfill normative, predictive, and structural–analytic roles with the aid of the theory of probability. This text then examines the study of labor market structures using analysis of job and career mobility, which is one of the approaches taken by sociologists in research on the labor market. Other chapters consider the characteristic trends and patterns from data on divorces. This book discusses as well the two approaches of stochastic modeling of social processes, namely competing risk models and semi-Markov processes. The final chapter deals with the practical application of regression models of survival data. This book is a valuable resource for social scientists and statisticians.

This book offers a systematic introduction to the optimal stochastic control theory via the dynamic programming principle, which is a powerful tool to analyze control problems. First we consider completely observable control problems with finite horizons. Using a time discretization we construct a nonlinear semigroup related to the dynamic programming principle (DPP), whose generator provides the Hamilton–Jacobi–Bellman (HJB) equation, and we characterize the value function via the nonlinear semigroup, besides the viscosity solution theory. When we control not only the dynamics of a system but also the terminal time of its evolution, control-stopping problems arise. This problem is treated in the same frameworks, via the nonlinear semigroup. Its results are applicable to the American option price problem. Zero-sum two-player time-homogeneous stochastic differential games and viscosity solutions of the Isaacs equations arising from such games are studied via a nonlinear semigroup related to DPP (the min-max principle, to be precise). Using semi-discretization arguments, we construct the nonlinear semigroups whose generators provide lower and upper Isaacs equations. Concerning partially observable control problems, we refer to stochastic parabolic equations driven by colored Wiener noises, in particular, the Zakai equation. The existence and uniqueness of solutions and regularities as well as Itô's formula are stated. A control problem for the Zakai equations has a nonlinear semigroup whose generator provides the HJB equation on a Banach space. The value function turns out to be a unique viscosity solution for the HJB equation under mild conditions. This edition provides a more generalized treatment of the topic than does the earlier book Lectures on Stochastic Control Theory (ISI Lecture Notes 9), where time-homogeneous cases are dealt with. Here, for finite time-horizon control problems, DPP was formulated as a one-parameter nonlinear semigroup, whose generator provides the HJB equation, by using a time-discretization method. The semigroup corresponds to the value function and is characterized as the envelope of Markovian transition semigroups of responses for constant control processes. Besides finite time-horizon controls, the book discusses control-stopping problems in the same frameworks.

In The Economics of Inaction, leading economist Nancy Stokey shows how the tools of stochastic control can be applied to dynamic problems of decision making under uncertainty when fixed costs are present. Stokey provides a self-contained, rigorous, and clear treatment of two types of models, impulse and instantaneous control. She presents the relevant results about Brownian motion and other diffusion processes, develops methods for analyzing each type of problem, and discusses applications to price setting, investment, and durable goods purchases."--Pub. desc.

Applied Stochastic Models and Control for Finance and Insurance presents at an introductory level some essential stochastic models applied in economics, finance and insurance. Markov chains, random walks, stochastic differential equations and other stochastic processes are used throughout the book and systematically applied to economic and financial applications. In addition, a dynamic programming framework is used to deal with some basic optimization problems. The book begins by introducing problems of economics, finance and insurance which involve time, uncertainty and risk. A number of cases are treated in detail, spanning risk management, volatility, memory, the time structure of preferences, interest rates and yields, etc. The second and third chapters provide an introduction to stochastic models and their application. Stochastic differential equations and stochastic calculus are presented in an intuitive manner, and numerous applications and exercises are used to facilitate their understanding and their use in Chapter 3. A number of other processes which are increasingly used in finance and insurance are introduced in Chapter 4. In the fifth chapter, ARCH and GARCH models are presented and their application to modeling volatility is emphasized. An outline of decision-making procedures is presented in Chapter 6. Furthermore, we also introduce the essentials of stochastic dynamic programming and control, and provide first steps for the student who seeks to apply these techniques. Finally, in Chapter 7, numerical techniques and approximations to stochastic processes are examined. This book can be used in business, economics, financial engineering and decision sciences schools for second year Master's students, as well as in a number of courses widely given in departments of statistics, systems and decision sciences.