The book presents a coherent treatment of Markov random walks and Markov additive processes together with their applications. Part I provides the foundations of these stochastic processes underpinned by a solid theoretical framework based on Semiregenerative phenomena. Part II presents some applications to queueing and storage systems.

The book is devoted to studies of quasi-stationary phenomena in nonlinearly perturbed stochastic systems. New methods of asymptotic analysis for nonlinearly perturbed stochastic processes based on new types of asymptotic expansions for perturbed renewal equation and recurrence algorithms for construction of asymptotic expansions for Markov type processes with absorption are presented. Asymptotic expansions are given in mixed ergodic (for processes) and large deviation theorems (for absorption times) for nonlinearly perturbed regenerative processes, semi-Markov processes, and Markov chains. Applications to analysis of quasi-stationary phenomena in nonlinearly perturbed queueing systems, population dynamics and epidemic models, and for risk processes are presented. The book also contains an extended bibliography of works in the area. It is an essential reference for theoretical and applied researchers in the field of stochastic processes and their applications and may be also useful for doctoral and advanced undergraduate students.

Probabilistic models are widely used for description and an analysis of various processes in system reliability, risk, queuing, data communication, logistic and storage systems. The book contains various applications of the theory of continuous-time Markov-modulated processes in operation research. All analytical results are illustrated by numerical computations. Used algorithms allow overcoming computation difficulties successfully. For example, a calculation of transient probabilities of states for a continuous-time finite Markov chain uses eigenvalues and eigenvectors of the corresponding matrix (generator). In a more complex case of differential or integral equations, such a simple explicit form of a solution is missing. The explicit form of solution is presented by means of infinity sums of functions. For example, often we have to deal with the so-called renewal equation. Its solution is presented as an infinite sum of the renewal function. In this case, an approximation of functions of interest and iterative computation procedures are used.

The book presents a coherent treatment of Markov random walks and Markov additive processes together with their applications. Part I provides the foundations of these stochastic processes underpinned by a solid theoretical framework based on Semiregenerative phenomena. Part II presents some applications to queueing and storage systems.

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader. - Presents both the theory and applications of the different aspects of Markov processes - Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented - Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.

This book highlights the latest advances in stochastic processes, probability theory, mathematical statistics, engineering mathematics and algebraic structures, focusing on mathematical models, structures, concepts, problems and computational methods and algorithms important in modern technology, engineering and natural sciences applications. It comprises selected, high-quality, refereed contributions from various large research communities in modern stochastic processes, algebraic structures and their interplay and applications. The chapters cover both theory and applications, illustrated by numerous figures, schemes, algorithms, tables and research results to help readers understand the material and develop new mathematical methods, concepts and computing applications in the future. Presenting new methods and results, reviews of cutting-edge research, and open problems and directions for future research, the book serves as a source of inspiration for a broad spectrum of researchers and research students in probability theory and mathematical statistics, applied algebraic structures, applied mathematics and other areas of mathematics and applications of mathematics. The book is based on selected contributions presented at the International Conference on “Stochastic Processes and Algebraic Structures – From Theory Towards Applications” (SPAS2017) to mark Professor Dmitrii Silvestrov’s 70th birthday and his 50 years of fruitful service to mathematics, education and international cooperation, which was held at Mälardalen University in Västerås and Stockholm University, Sweden, in October 2017.