This thoroughly revised and expanded new edition now includes a more detailed treatment of the EM algorithm, a description of an efficient approximate Viterbi-training procedure, a theoretical derivation of the perplexity measure and coverage of multi-pass decoding based on n-best search. Supporting the discussion of the theoretical foundations of Markov modeling, special emphasis is also placed on practical algorithmic solutions. Features: introduces the formal framework for Markov models; covers the robust handling of probability quantities; presents methods for the configuration of hidden Markov models for specific application areas; describes important methods for efficient processing of Markov models, and the adaptation of the models to different tasks; examines algorithms for searching within the complex solution spaces that result from the joint application of Markov chain and hidden Markov models; reviews key applications of Markov models.

The aim of this book volume is to explain the importance of Markov state models to molecular simulation, how they work, and how they can be applied to a range of problems. The Markov state model (MSM) approach aims to address two key challenges of molecular simulation: 1) How to reach long timescales using short simulations of detailed molecular models. 2) How to systematically gain insight from the resulting sea of data. MSMs do this by providing a compact representation of the vast conformational space available to biomolecules by decomposing it into states sets of rapidly interconverting conformations and the rates of transitioning between states. This kinetic definition allows one to easily vary the temporal and spatial resolution of an MSM from high-resolution models capable of quantitative agreement with (or prediction of) experiment to low-resolution models that facilitate understanding. Additionally, MSMs facilitate the calculation of quantities that are difficult to obtain from more direct MD analyses, such as the ensemble of transition pathways. This book introduces the mathematical foundations of Markov models, how they can be used to analyze simulations and drive efficient simulations, and some of the insights these models have yielded in a variety of applications of molecular simulation.

Here is a work that adds much to the sum of our knowledge in a key area of science today. It is concerned with the estimation of discrete-time semi-Markov and hidden semi-Markov processes. A unique feature of the book is the use of discrete time, especially useful in some specific applications where the time scale is intrinsically discrete. The models presented in the book are specifically adapted to reliability studies and DNA analysis. The book is mainly intended for applied probabilists and statisticians interested in semi-Markov chains theory, reliability and DNA analysis, and for theoretical oriented reliability and bioinformatics engineers.

As more applications are found, interest in Hidden Markov Models continues to grow. Following comments and feedback from colleagues, students and other working with Hidden Markov Models the corrected 3rd printing of this volume contains clarifications, improvements and some new material, including results on smoothing for linear Gaussian dynamics. In Chapter 2 the derivation of the basic filters related to the Markov chain are each presented explicitly, rather than as special cases of one general filter. Furthermore, equations for smoothed estimates are given. The dynamics for the Kalman filter are derived as special cases of the authors’ general results and new expressions for a Kalman smoother are given. The Chapters on the control of Hidden Markov Chains are expanded and clarified. The revised Chapter 4 includes state estimation for discrete time Markov processes and Chapter 12 has a new section on robust control.

A number of methodologies have been employed to provide decision making solutions globalized markets. Hidden Markov Models in Finance offers the first systematic application of these methods to specialized financial problems: option pricing, credit risk modeling, volatility estimation and more. The book provides tools for sorting through turbulence, volatility, emotion, chaotic events – the random "noise" of financial markets – to analyze core components.

This book is a comprehensive treatment of inference for hidden Markov models, including both algorithms and statistical theory. Topics range from filtering and smoothing of the hidden Markov chain to parameter estimation, Bayesian methods and estimation of the number of states. In a unified way the book covers both models with finite state spaces and models with continuous state spaces (also called state-space models) requiring approximate simulation-based algorithms that are also described in detail. Many examples illustrate the algorithms and theory. This book builds on recent developments to present a self-contained view.

This book presents a radically new approach to problems of evaluating and optimizing the performance of continuous-time stochastic systems. This approach is based on the use of a family of Markov processes called Piecewise-Deterministic Processes (PDPs) as a general class of stochastic system models. A PDP is a Markov process that follows deterministic trajectories between random jumps, the latter occurring either spontaneously, in a Poisson-like fashion, or when the process hits the boundary of its state space. This formulation includes an enormous variety of applied problems in engineering, operations research, management science and economics as special cases; examples include queueing systems, stochastic scheduling, inventory control, resource allocation problems, optimal planning of production or exploitation of renewable or non-renewable resources, insurance analysis, fault detection in process systems, and tracking of maneuvering targets, among many others. The first part of the book shows how these applications lead to the PDP as a system model, and the main properties of PDPs are derived. There is particular emphasis on the so-called extended generator of the process, which gives a general method for calculating expectations and distributions of system performance functions. The second half of the book is devoted to control theory for PDPs, with a view to controlling PDP models for optimal performance: characterizations are obtained of optimal strategies both for continuously-acting controllers and for control by intervention (impulse control). Throughout the book, modern methods of stochastic analysis are used, but all the necessary theory is developed from scratch and presented in a self-contained way. The book will be useful to engineers and scientists in the application areas as well as to mathematicians interested in applications of stochastic analysis.