A number of processes that occur in nature as well as those that are manifestations of human activities are correlated in nature and can be describe by stochastic non-Markovian processes. Most known theoretical results for these systems are in the steady state domain, assuming that the system has been in operation for a long enough time, and that the state in which the system starts has no effect on the current behavior of the system. Nevertheless steady state assumptions do not hold in many applied situations. In this thesis we provide a framework to stochastically track these processes. Application of this theory provide valuable insights into the transient behavior of these stochastic processes and allows us to model and study the effect of auto-correlations in the driving processes on transient probabilistic (performance) metrics of interest. In order to develop accurate models to represent these systems, we allow the arrival and the service processes that characterize the system to be both general and correlated. We specifically study the busy period and other first passages of and auto-correlated MEP/MEP/1 single server queue to demonstrate the application of tracking these memory-full processes. Analysis presented here is the transient domain and does not require the underlying processes to be in a steady state. In the first part of the thesis we provide solutions to compute the probabilities for exactly 'n' customers being served in a busy period of MEP/MEP/1 queueing system. We then present matrix exponential representations to characterize the lengths of sample paths during these busy periods and derive expressions to compute moments for length of the busy period as well as for the number of customers served during the busy period. In the second part of the thesis, we study the effect of increase in threshold level and the correlations in the arrival and service processes on the mean first passage time to go below a given threshold. Finally we study the busy periods for finite queueing systems, and again study both the length of the busy period and the number of customers served during such a time.

Sample-Path Analysis of Queueing Systems uses a deterministic (sample-path) approach to analyze stochastic systems, primarily queueing systems and more general input-output systems. Among other topics of interest it deals with establishing fundamental relations between asymptotic frequencies and averages, pathwise stability, and insensitivity. These results are utilized to establish useful performance measures. The intuitive deterministic approach of this book will give researchers, teachers, practitioners, and students better insights into many results in queueing theory. The simplicity and intuitive appeal of the arguments will make these results more accessible, with no sacrifice of mathematical rigor. Recent topics such as pathwise stability are also covered in this context. The book consistently takes the point of view of focusing on one sample path of a stochastic process. Hence, it is devoted to providing pure sample-path arguments. With this approach it is possible to separate the issue of the validity of a relationship from issues of existence of limits and/or construction of stationary framework. Generally, in many cases of interest in queueing theory, relations hold, assuming limits exist, and the proofs are elementary and intuitive. In other cases, proofs of the existence of limits will require the heavy machinery of stochastic processes. The authors feel that sample-path analysis can be best used to provide general results that are independent of stochastic assumptions, complemented by use of probabilistic arguments to carry out a more detailed analysis. This book focuses on the first part of the picture. It does however, provide numerous examples that invoke stochastic assumptions, which typically are presented at the ends of the chapters.

Building on the author’s more than 35 years of teaching experience, Modeling and Analysis of Stochastic Systems, Third Edition, covers the most important classes of stochastic processes used in the modeling of diverse systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models. The third edition has been updated with several new applications, including the Google search algorithm in discrete time Markov chains, several examples from health care and finance in continuous time Markov chains, and square root staffing rule in Queuing models. More than 50 new exercises have been added to enhance its use as a course text or for self-study. The sequence of chapters and exercises has been maintained between editions, to enable those now teaching from the second edition to use the third edition. Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, readers will be well-equipped to build and analyze useful stochastic models for real-life situations.

This book gives a complete and elementary account of fundamental results on hyperfinite measures and their application to stochastic processes, including the *-finite Stieltjes sum approximation of martingale integrals. Many detailed examples, not found in the literature, are included. It begins with a brief chapter on tools from logic and infinitesimal (or non-standard) analysis so that the material is accessible to beginning graduate students.

An Introduction to Stochastic Modeling provides information pertinent to the standard concepts and methods of stochastic modeling. This book presents the rich diversity of applications of stochastic processes in the sciences. Organized into nine chapters, this book begins with an overview of diverse types of stochastic models, which predicts a set of possible outcomes weighed by their likelihoods or probabilities. This text then provides exercises in the applications of simple stochastic analysis to appropriate problems. Other chapters consider the study of general functions of independent, identically distributed, nonnegative random variables representing the successive intervals between renewals. This book discusses as well the numerous examples of Markov branching processes that arise naturally in various scientific disciplines. The final chapter deals with queueing models, which aid the design process by predicting system performance. This book is a valuable resource for students of engineering and management science. Engineers will also find this book useful.

This book provides an introductory albeit solid presentation of path integration techniques as applied to the field of stochastic processes. The subject began with the work of Wiener during the 1920''s, corresponding to a sum over random trajectories, anticipating by two decades Feynman''s famous work on the path integral representation of quantum mechanics. However, the true trigger for the application of these techniques within nonequilibrium statistical mechanics and stochastic processes was the work of Onsager and Machlup in the early 1950''s. The last quarter of the 20th century has witnessed a growing interest in this technique and its application in several branches of research, even outside physics (for instance, in economy).The aim of this book is to offer a brief but complete presentation of the path integral approach to stochastic processes. It could be used as an advanced textbook for graduate students and even ambitious undergraduates in physics. It describes how to apply these techniques for both Markov and non-Markov processes. The path expansion (or semiclassical approximation) is discussed and adapted to the stochastic context. Also, some examples of nonlinear transformations and some applications are discussed, as well as examples of rather unusual applications. An extensive bibliography is included. The book is detailed enough to capture the interest of the curious reader, and complete enough to provide a solid background to explore the research literature and start exploiting the learned material in real situations.