Stochastic Recursive Algorithms for Optimization presents algorithms for constrained and unconstrained optimization and for reinforcement learning. Efficient perturbation approaches form a thread unifying all the algorithms considered. Simultaneous perturbation stochastic approximation and smooth fractional estimators for gradient- and Hessian-based methods are presented. These algorithms: • are easily implemented; • do not require an explicit system model; and • work with real or simulated data. Chapters on their application in service systems, vehicular traffic control and communications networks illustrate this point. The book is self-contained with necessary mathematical results placed in an appendix. The text provides easy-to-use, off-the-shelf algorithms that are given detailed mathematical treatment so the material presented will be of significant interest to practitioners, academic researchers and graduate students alike. The breadth of applications makes the book appropriate for reader from similarly diverse backgrounds: workers in relevant areas of computer science, control engineering, management science, applied mathematics, industrial engineering and operations research will find the content of value.

Stochastic Recursive Algorithms for Optimization presents algorithms for constrained and unconstrained optimization and for reinforcement learning. Efficient perturbation approaches form a thread unifying all the algorithms considered. Simultaneous perturbation stochastic approximation and smooth fractional estimators for gradient- and Hessian-based methods are presented. These algorithms: • are easily implemented; • do not require an explicit system model; and • work with real or simulated data. Chapters on their application in service systems, vehicular traffic control and communications networks illustrate this point. The book is self-contained with necessary mathematical results placed in an appendix. The text provides easy-to-use, off-the-shelf algorithms that are given detailed mathematical treatment so the material presented will be of significant interest to practitioners, academic researchers and graduate students alike. The breadth of applications makes the book appropriate for reader from similarly diverse backgrounds: workers in relevant areas of computer science, control engineering, management science, applied mathematics, industrial engineering and operations research will find the content of value.

This book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. This second edition is a thorough revision, although the main features and structure remain unchanged. It contains many additional applications and results as well as more detailed discussion.

* Unique in its survey of the range of topics. * Contains a strong, interdisciplinary format that will appeal to both students and researchers. * Features exercises and web links to software and data sets.

This book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. This second edition is a thorough revision, although the main features and structure remain unchanged. It contains many additional applications and results as well as more detailed discussion.

The DMV seminar "Stochastische Approximation und Optimierung zufalliger Systeme" was held at Blaubeuren, 28. 5. -4. 6. 1989. The goal was to give an approach to theory and application of stochas tic approximation in view of optimization problems, especially in engineering systems. These notes are based on the seminar lectures. They consist of three parts: I. Foundations of stochastic approximation (H. Walk); n. Applicational aspects of stochastic approximation (G. PHug); In. Applications to adaptation :ugorithms (L. Ljung). The prerequisites for reading this book are basic knowledge in probability, mathematical statistics, optimization. We would like to thank Prof. M. Barner and Prof. G. Fischer for the or ganization of the seminar. We also thank the participants for their cooperation and our assistants and secretaries for typing the manuscript. November 1991 L. Ljung, G. PHug, H. Walk Table of contents I Foundations of stochastic approximation (H. Walk) §1 Almost sure convergence of stochastic approximation procedures 2 §2 Recursive methods for linear problems 17 §3 Stochastic optimization under stochastic constraints 22 §4 A learning model; recursive density estimation 27 §5 Invariance principles in stochastic approximation 30 §6 On the theory of large deviations 43 References for Part I 45 11 Applicational aspects of stochastic approximation (G. PHug) §7 Markovian stochastic optimization and stochastic approximation procedures 53 §8 Asymptotic distributions 71 §9 Stopping times 79 §1O Applications of stochastic approximation methods 80 References for Part II 90 III Applications to adaptation algorithms (L.