This book is an integrated work published in two volumes. The first volume treats the basic Markov process and its variants; the second, semi-Markov and decision processes. Its intent is to equip readers to formulate, analyze, and evaluate simple and advanced Markov models of systems, ranging from genetics and space engineering to marketing. More than a collection of techniques, it constitutes a guide to the consistent application of the fundamental principles of probability and linear system theory. Author Ronald A. Howard, Professor of Management Science and Engineering at Stanford University, continues his treatment from Volume I with surveys of the discrete- and continuous-time semi-Markov processes, continuous-time Markov processes, and the optimization procedure of dynamic programming. The final chapter reviews the preceding material, focusing on the decision processes with discussions of decision structure, value and policy iteration, and examples of infinite duration and transient processes. Volume II concludes with an appendix listing the properties of congruent matrix multiplication.

This book is an integrated work published in two volumes. The first volume treats the basic Markov process and its variants; the second, semi-Markov and decision processes. Its intent is to equip readers to formulate, analyze, and evaluate simple and advanced Markov models of systems, ranging from genetics and space engineering to marketing. More than a collection of techniques, it constitutes a guide to the consistent application of the fundamental principles of probability and linear system theory. Author Ronald A. Howard, Professor of Management Science and Engineering at Stanford University, begins with the basic Markov model, proceeding to systems analyses of linear processes and Markov processes, transient Markov processes and Markov process statistics, and statistics and inference. Subsequent chapters explore recurrent events and random walks, Markovian population models, and time-varying Markov processes. Volume I concludes with a pair of helpful indexes.

This book constitutes the refereed proceedings of the Fifth International AMAST Workshop on Formal Methods for Real-Time and Probabilistic Systems, ARTS '99, held in Bamberg, Germany in May 1999. The 17 revised full papers presented together with three invited contributions were carefully reviewed and selected from 33 submissions. The papers are organized in topical sections on verification of probabilistic systems, model checking for probabilistic systems, semantics of probabilistic process calculi, semantics of real-time processes, real-time compilation, stochastic process algebra, and modeling and verification of real-time systems.

This book is aimed at both researchers and practitioners, and provides a collection of expert systems in manufacturing and production engineering along with their knowledge base and rules. We believe that inclusion of the knowledge base and associated rules is essential if practitioners are to derive full benefit from these expert systems. This unique book is the result of our belief and the efforts of our distinguished colleagues who subscribe to this philosophy. A total of 15 different expert systems are included in this book. These expert systems are preceded by an introductory chapter written by Kuo, Preface XVll Mital and Anand. The expert system rules are included on a floppy disk in ASCII and can be easily accessed. These rules and the description of the expert system's structure should assist the users in customizing these systems. Overall, the expert systems included in this volume cover a fairly wide variety of manufacturing and production engineering topics.

Ernst G. Frankel This book has its origin in lecture notes developed over several years for use in a course in Systems Reliability for engineers concerned with the design of physical systems such as civil structures, power plants, and transport vehicles of all types. Increasing public concern with the reliability o~ systems for reasons of human safety, environmental protection, and acceptable ir. vestment risk limitations has resulted in an increasing interest by engineers in the formal applica~i0n of reliability theory to e~gineering desian. At the same time there is a demand for more effective approaches to the des~gn of procedures for the operation and use of man-made syste~s and more meaningful assessment of the risks intr)duction and use of such a system poses both when operating as designed and when operating at below design performance. The purpose of the book is to provide a sound, yet practical, introduction to reliability analysis and risk assessment which can be used by professionals in engineering, planning, management, and economics to improve the design, operation, and risk assessment of systems of interest. The text should be useful for students in many disciplines and is designed for fourth~year undergraduates or first-year graduate students. I would like to acknowledge the help of many of my graduate students who contributed to the development of this book by offering comments and criticism. Similarly I would like to thank Mrs.

Traditionally, models and methods for the analysis of the functional correctness of reactive systems, and those for the analysis of their performance (and - pendability) aspects, have been studied by di?erent research communities. This has resulted in the development of successful, but distinct and largely unrelated modeling and analysis techniques for both domains. In many modern systems, however, the di?erence between their functional features and their performance properties has become blurred, as relevant functionalities become inextricably linked to performance aspects, e.g. isochronous data transfer for live video tra- mission. During the last decade, this trend has motivated an increased interest in c- bining insights and results from the ?eld of formal methods – traditionally - cused on functionality – with techniques for performance modeling and analysis. Prominent examples of this cross-fertilization are extensions of process algebra and Petri nets that allow for the automatic generation of performance models, the use of formal proof techniques to assess the correctness of randomized - gorithms, and extensions of model checking techniques to analyze performance requirements automatically. We believe that these developments markthe - ginning of a new paradigm for the modeling and analysis of systems in which qualitative and quantitative aspects are studied from an integrated perspective. We are convinced that the further worktowards the realization of this goal will be a growing source of inspiration and progress for both communities.

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader. - Presents both the theory and applications of the different aspects of Markov processes - Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented - Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.