This book presents a concise introduction to piecewise deterministic Markov processes (PDMPs), with particular emphasis on their applications to biological models. Further, it presents examples of biological phenomena, such as gene activity and population growth, where different types of PDMPs appear: continuous time Markov chains, deterministic processes with jumps, processes with switching dynamics, and point processes. Subsequent chapters present the necessary tools from the theory of stochastic processes and semigroups of linear operators, as well as theoretical results concerning the long-time behaviour of stochastic semigroups induced by PDMPs and their applications to biological models. As such, the book offers a valuable resource for mathematicians and biologists alike. The first group will find new biological models that lead to interesting and often new mathematical questions, while the second can observe how to include seemingly disparate biological proc esses into a unified mathematical theory, and to arrive at revealing biological conclusions. The target audience primarily comprises of researchers in these two fields, but the book will also benefit graduate students.

Mathematically rigorous exposition of the basic theory of marked point processes and piecewise deterministic stochastic processes Point processes are constructed from scratch with detailed proofs Includes applications with examples and exercises in survival analysis, branching processes, ruin probabilities, sports (soccer), finance and risk management, and queueing theory Accessible to a wider cross-disciplinary audience

The theory of Markov decision processes focuses on controlled Markov chains in discrete time. The authors establish the theory for general state and action spaces and at the same time show its application by means of numerous examples, mostly taken from the fields of finance and operations research. By using a structural approach many technicalities (concerning measure theory) are avoided. They cover problems with finite and infinite horizons, as well as partially observable Markov decision processes, piecewise deterministic Markov decision processes and stopping problems. The book presents Markov decision processes in action and includes various state-of-the-art applications with a particular view towards finance. It is useful for upper-level undergraduates, Master's students and researchers in both applied probability and finance, and provides exercises (without solutions).

This book presents a radically new approach to problems of evaluating and optimizing the performance of continuous-time stochastic systems. This approach is based on the use of a family of Markov processes called Piecewise-Deterministic Processes (PDPs) as a general class of stochastic system models. A PDP is a Markov process that follows deterministic trajectories between random jumps, the latter occurring either spontaneously, in a Poisson-like fashion, or when the process hits the boundary of its state space. This formulation includes an enormous variety of applied problems in engineering, operations research, management science and economics as special cases; examples include queueing systems, stochastic scheduling, inventory control, resource allocation problems, optimal planning of production or exploitation of renewable or non-renewable resources, insurance analysis, fault detection in process systems, and tracking of maneuvering targets, among many others. The first part of the book shows how these applications lead to the PDP as a system model, and the main properties of PDPs are derived. There is particular emphasis on the so-called extended generator of the process, which gives a general method for calculating expectations and distributions of system performance functions. The second half of the book is devoted to control theory for PDPs, with a view to controlling PDP models for optimal performance: characterizations are obtained of optimal strategies both for continuously-acting controllers and for control by intervention (impulse control). Throughout the book, modern methods of stochastic analysis are used, but all the necessary theory is developed from scratch and presented in a self-contained way. The book will be useful to engineers and scientists in the application areas as well as to mathematicians interested in applications of stochastic analysis.

This book is a collective volume authored by leading scientists in the field of stochastic modelling, associated statistical topics and corresponding applications. The main classes of stochastic processes for dependent data investigated throughout this book are Markov, semi-Markov, autoregressive and piecewise deterministic Markov models. The material is divided into three parts corresponding to: (i) Markov and semi-Markov processes, (ii) autoregressive processes and (iii) techniques based on divergence measures and entropies. A special attention is payed to applications in reliability, survival analysis and related fields.

Labelled Markov processes are probabilistic versions of labelled transition systems with continuous state spaces. The book covers basic probability and measure theory on continuous state spaces and then develops the theory of LMPs.

Piecewise-deterministic Markov processes form a class of stochastic models with a sizeable scope of applications: biology, insurance, neuroscience, networks, finance... Such processes are defined by a deterministic motion punctuated by random jumps at random times, and offer simple yet challenging models to study. Nevertheless, the issue of statistical estimation of the parameters ruling the jump mechanism is far from trivial. Responding to new developments in the field as well as to current research interests and needs, Statistical inference for piecewise-deterministic Markov processes offers a detailed and comprehensive survey of state-of-the-art results. It covers a wide range of general processes as well as applied models. The present book also dwells on statistics in the context of Markov chains, since piecewise-deterministic Markov processes are characterized by an embedded Markov chain corresponding to the position of the process right after the jumps.