The stationary conditional doubly limiting conditional and limiting conditional mean ratio quasi-stationary distributions are given for continuous time Markov Chains with denumerable state space. (Author).

Main concepts of quasi-stationary distributions (QSDs) for killed processes are the focus of the present volume. For diffusions, the killing is at the boundary and for dynamical systems there is a trap. The authors present the QSDs as the ones that allow describing the long-term behavior conditioned to not being killed. Studies in this research area started with Kolmogorov and Yaglom and in the last few decades have received a great deal of attention. The authors provide the exponential distribution property of the killing time for QSDs, present the more general result on their existence and study the process of trajectories that survive forever. For birth-and-death chains and diffusions, the existence of a single or a continuum of QSDs is described. They study the convergence to the extremal QSD and give the classification of the survival process. In this monograph, the authors discuss Gibbs QSDs for symbolic systems and absolutely continuous QSDs for repellers. The findings described are relevant to researchers in the fields of Markov chains, diffusions, potential theory, dynamical systems, and in areas where extinction is a central concept. The theory is illustrated with numerous examples. The volume uniquely presents the distribution behavior of individuals who survive in a decaying population for a very long time. It also provides the background for applications in mathematical ecology, statistical physics, computer sciences, and economics.

This book presents an algebraic development of the theory of countable state space Markov chains with discrete- and continuous-time parameters. A Markov chain is a stochastic process characterized by the Markov prop erty that the distribution of future depends only on the current state, not on the whole history. Despite its simple form of dependency, the Markov property has enabled us to develop a rich system of concepts and theorems and to derive many results that are useful in applications. In fact, the areas that can be modeled, with varying degrees of success, by Markov chains are vast and are still expanding. The aim of this book is a discussion of the time-dependent behavior, called the transient behavior, of Markov chains. From the practical point of view, when modeling a stochastic system by a Markov chain, there are many instances in which time-limiting results such as stationary distributions have no meaning. Or, even when the stationary distribution is of some importance, it is often dangerous to use the stationary result alone without knowing the transient behavior of the Markov chain. Not many books have paid much attention to this topic, despite its obvious importance.

This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.

The book is devoted to studies of quasi-stationary phenomena in nonlinearly perturbed stochastic systems. New methods of asymptotic analysis for nonlinearly perturbed stochastic processes based on new types of asymptotic expansions for perturbed renewal equation and recurrence algorithms for construction of asymptotic expansions for Markov type processes with absorption are presented. Asymptotic expansions are given in mixed ergodic (for processes) and large deviation theorems (for absorption times) for nonlinearly perturbed regenerative processes, semi-Markov processes, and Markov chains. Applications to analysis of quasi-stationary phenomena in nonlinearly perturbed queueing systems, population dynamics and epidemic models, and for risk processes are presented. The book also contains an extended bibliography of works in the area. It is an essential reference for theoretical and applied researchers in the field of stochastic processes and their applications and may be also useful for doctoral and advanced undergraduate students.

Many Markov chains with a single absorbing state have a unique limiting conditional distribution (LCD) to which they converge, conditioned on non-absorption, regardless of the initial distribution. If this limiting conditional distribution is used as the initial distribution over the non-absorbing states, then the probability distribution of the process at time n, conditioned on non-absorption, is equal for all values of n>0. Such an initial distribution is known as the quasi-stationary distribution (QSD). Thus the LCD and QSD are equal. These distributions can be found in both the discrete-time and continuous-time case. In this thesis we consider finite Markov chains which have one absorbing state, and for which all other states form a set which is a single communicating class. In addition, every state is aperiodic. These conditions ensure the existence of a unique LCD. We first consider continuous Markov chains in the context of survival analysis. We consider the hazard rate, a function which measures the risk of instantaneous failure of a system at time t conditioned on the system not having failed before t. It is well-known that the QSD leads to a constant hazard rate, and that the hazard rate generated by any other initial distribution tends to that constant rate. Claims have been made by Aalen and by Aalen and Gjessing that it may be possible to predict the shape of hazard rates generated by phase type distributions (first passage time distributions generated by atomic initial distributions) by comparing these initial distributions with the QSD. In Chapter 2 we consider these claims, and demonstrate through the use of several examples that the behaviour considered by those conjectures is more complex then previously believed. In Chapters 3 and 4 we consider discrete Markov chains in the context of imprecise probability. In many situations it may be unrealistic to assume that the transition matrix of a Markov chain can be determined exactly. It may be more plausible to determine upper and lower bounds upon each element, or even determine closed sets of probability distributions to which the rows of the matrix may belong. Such methods have been discussed by Kozine and Utkin and by Skulj, and in each of these papers results were given regarding the long-term behaviour of such processes. None of these papers considered Markov chains with an absorbing state. In Chapter 3 we demonstrate that, under the assumption that the transition matrix cannot change from time step to time step, there exists an imprecise generalisation to both the LCD and the QSD, and that these two generalisations are equal. In Chapter 4, we prove that this result holds even when we no longer assume that the transition matrix cannot change from time step to time step. In each chapter, examples are presented demonstrating the convergence of such processes, and Chapter 4 includes a comparison between the two methods.