Intersecting two large research areas - numerical analysis and applied probability/queuing theory - this book is a self-contained introduction to the numerical solution of structured Markov chains, which have a wide applicability in queuing theory and stochastic modeling and include M/G/1 and GI/M/1-type Markov chain, quasi-birth-death processes, non-skip free queues and tree-like stochastic processes. Written for applied probabilists and numerical analysts, but accessible toengineers and scientists working on telecommunications and evaluation of computer systems performances, it provides a systematic treatment of the theory and algorithms for important families of structured Markov chains and a thorough overview of the current literature.The book, consisting of nine Chapters, is presented in three parts. Part 1 covers a basic description of the fundamental concepts related to Markov chains, a systematic treatment of the structure matrix tools, including finite Toeplitz matrices, displacement operators, FFT, and the infinite block Toeplitz matrices, their relationship with matrix power series and the fundamental problems of solving matrix equations and computing canonical factorizations. Part 2 deals with the description andanalysis of structure Markov chains and includes M/G/1, quasi-birth-death processes, non-skip-free queues and tree-like processes. Part 3 covers solution algorithms where new convergence and applicability results are proved. Each chapter ends with bibliographic notes for further reading, and the bookends with an appendix collecting the main general concepts and results used in the book, a list of the main annotations and algorithms used in the book, and an extensive index.

Markov Chains -- Direct Methods -- Iterative Methods -- Projection Methods -- Block Hessenberg Matrices -- Decompositional Methods -- LI-Cyclic Markov -- Chains -- Transient Solutions -- Stochastic Automata Networks -- Software.

This graduate textbook provides an alternative to discrete event simulation. It describes how to formulate discrete event systems, how to convert them into Markov chains, and how to calculate their transient and equilibrium probabilities. The most appropriate methods for finding these probabilities are described in some detail, and templates for efficient algorithms are provided. These algorithms can be executed on any laptop, even in cases where the Markov chain has hundreds of thousands of states. This book features the probabilistic interpretation of Gaussian elimination, a concept that unifies many of the topics covered, such as embedded Markov chains and matrix analytic methods. The material provided should aid practitioners significantly to solve their problems. This book also provides an interesting approach to teaching courses of stochastic processes.

Papers presented at a workshop held January 1990 (location unspecified) cover just about all aspects of solving Markov models numerically. There are papers on matrix generation techniques and generalized stochastic Petri nets; the computation of stationary distributions, including aggregation/disagg

Les chaînes de Markov sont des modèles probabilistes utilisés dans des domaines variés comme la logistique, l'informatique, la fiabilité, les télécommunications, ou encore la biologie et la physique-chimie. On les retrouve également dans la finance, l’économie et les sciences sociales. Cet ouvrage présente une étude approfondie des chaînes de Markov à temps discret et à temps continu avec des applications détaillées aux processus de naissance et mort et aux files d'attente. Ces applications sont illustrées par des algorithmes généraux de calcul de probabilités d'état et de distribution de temps de passage. Le développement de ces algorithmes repose sur l'utilisation de la technique d'uniformisation des chaînes de Markov qui est présentée de manière théorique et intuitive. Ce livre s'adresse aux ingénieurs et chercheurs ayant besoin de modèles probabilistes pour évaluer et prédire le comportement des systèmes qu'ils étudient ou qu'ils développent. Il est aussi très bien adapté pour un cours de master.

Matrix-analytic and related methods have become recognized as an important and fundamental approach for the mathematical analysis of general classes of complex stochastic models. Research in the area of matrix-analytic and related methods seeks to discover underlying probabilistic structures intrinsic in such stochastic models, develop numerical algorithms for computing functionals (e.g., performance measures) of the underlying stochastic processes, and apply these probabilistic structures and/or computational algorithms within a wide variety of fields. This volume presents recent research results on: the theory, algorithms and methodologies concerning matrix-analytic and related methods in stochastic models; and the application of matrix-analytic and related methods in various fields, which includes but is not limited to computer science and engineering, communication networks and telephony, electrical and industrial engineering, operations research, management science, financial and risk analysis, and bio-statistics. These research studies provide deep insights and understanding of the stochastic models of interest from a mathematics and/or applications perspective, as well as identify directions for future research.

This treatment of the basic theory of algebraic Riccati equations describes the classical as well as the more advanced algorithms for their solution in a manner that is accessible to both practitioners and scholars. It is the first book in which nonsymmetric algebraic Riccati equations are treated in a clear and systematic way. Some proofs of theoretical results have been simplified and a unified notation has been adopted. Readers will find a unified discussion of doubling algorithms, which are effective in solving algebraic Riccati equations as well as a detailed description of all classical and advanced algorithms for solving algebraic Riccati equations and their MATLAB codes. This will help the reader gain an understanding of the computational issues and provide ready-to-use implementation of the different solution techniques.