Stochastic control plays an important role in many scientific and applied disciplines including communications, engineering, medicine, finance and many others. It is one of the effective methods being used to find optimal decision-making strategies in applications. The book provides a collection of outstanding investigations in various aspects of stochastic systems and their behavior. The book provides a self-contained treatment on practical aspects of stochastic modeling and calculus including applications drawn from engineering, statistics, and computer science. Readers should be familiar with basic probability theory and have a working knowledge of stochastic calculus. PhD students and researchers in stochastic control will find this book useful.

This volume originates from two workshops, both focusing on themes that are reflected in the title of the volume. The first workshop took place at Eindhoven University of Technology, April 24-26, 2001, on the occasion of the University granting a doctorate honoris causa to Profes sor John A. Buzacott. The second workshop was held on June 15, 2002 at Cornell University (preceding the annual INFORMSjMSOM Confer ence), honoring John's retirement and his lifetime contributions. Each of the two workshops consisted of about a dozen technical presentations. The objective of the volume, however, is not to simply publish the proceedings of the two workshops. Rather, our objective is to put to gether a select set of articles, each organized into a well-written chapter, focusing on a timely topic. Collected into a single volume, these chapters aim to serve as a useful reference for researchers and practitioners alike, and also as reading materials for graduate courses or seminars.

Stochastic Models: Estimation and Control: v. 2

Serving as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus, Introduction to Stochastic Modeling, Fourth Edition, bridges the gap between basic probability and an intermediate level course in stochastic processes. The objectives of the text are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems. New to this edition: Realistic applications from a variety of disciplines integrated throughout the text, including more biological applications Plentiful, completely updated problems Completely updated and reorganized end-of-chapter exercise sets, 250 exercises with answers New chapters of stochastic differential equations and Brownian motion and related processes Additional sections on Martingale and Poisson process Realistic applications from a variety of disciplines integrated throughout the text Extensive end of chapter exercises sets, 250 with answers Chapter 1-9 of the new edition are identical to the previous edition New! Chapter 10 - Random Evolutions New! Chapter 11- Characteristic functions and Their Applications

This book is dedicated to Jinhua Cao on the occasion of his 80th birthday. Jinhua Cao is one of the most famous reliability theorists. His main contributions include: published over 100 influential scientific papers; published an interesting reliability book in Chinese in 1986, which has greatly influenced the reliability of education, academic research and engineering applications in China; initiated and organized Reliability Professional Society of China (the first part of Operations Research Society of China) since 1981. The high admiration that Professor Cao enjoys in the reliability community all over the world was witnessed by the enthusiastic response of each contributor in this book. The contributors are leading researchers with diverse research perspectives. The research areas of the book iclude a broad range of topics related to reliability models, queueing theory, manufacturing systems, supply chain finance, risk management, Markov decision processes, blockchain and so forth. The book consists of a brief Preface describing the main achievements of Professor Cao; followed by congratulations from Professors Way Kuo and Wei Wayne Li, and by Operations Research Society of China, and Reliability Professional Society of China; and further followed by 25 articles roughly grouped together. Most of the articles are written in a style understandable to a wide audience. This book is useful to anyone interested in recent developments in reliability, network security, system safety, and their stochastic modeling and analysis.

This sequel to volume 19 of Handbook on Statistics on Stochastic Processes: Modelling and Simulation is concerned mainly with the theme of reviewing and, in some cases, unifying with new ideas the different lines of research and developments in stochastic processes of applied flavour. This volume consists of 23 chapters addressing various topics in stochastic processes. These include, among others, those on manufacturing systems, random graphs, reliability, epidemic modelling, self-similar processes, empirical processes, time series models, extreme value therapy, applications of Markov chains, modelling with Monte Carlo techniques, and stochastic processes in subjects such as engineering, telecommunications, biology, astronomy and chemistry. particular with modelling, simulation techniques and numerical methods concerned with stochastic processes. The scope of the project involving this volume as well as volume 19 is already clarified in the preface of volume 19. The present volume completes the aim of the project and should serve as an aid to students, teachers, researchers and practitioners interested in applied stochastic processes.

Volume 1: Deterministic Modeling, Methods and Analysis For more than half a century, stochastic calculus and stochastic differential equations have played a major role in analyzing the dynamic phenomena in the biological and physical sciences, as well as engineering. The advancement of knowledge in stochastic differential equations is spreading rapidly across the graduate and postgraduate programs in universities around the globe. This will be the first available book that can be used in any undergraduate/graduate stochastic modeling/applied mathematics courses and that can be used by an interdisciplinary researcher with a minimal academic background. An Introduction to Differential Equations: Volume 2 is a stochastic version of Volume 1 (“An Introduction to Differential Equations: Deterministic Modeling, Methods and Analysis”). Both books have a similar design, but naturally, differ by calculi. Again, both volumes use an innovative style in the presentation of the topics, methods and concepts with adequate preparation in deterministic Calculus. Errata Errata (32 KB)

A comprehensive up-to-date presentation of some of the classical areas of reliability, based on a more advanced probabilistic framework using the modern theory of stochastic processes. This framework allows analysts to formulate general failure models, establish formulae for computing various performance measures, as well as determine how to identify optimal replacement policies in complex situations.

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader. - Presents both the theory and applications of the different aspects of Markov processes - Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented - Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.

In geodesy and geoinformation science, as well as in many other technical disciplines, it is often not possible to directly determine the desired target quantities. Therefore, the unknown parameters must be linked with the measured values by a mathematical model which consists of the functional and the stochastic models. The functional model describes the geometrical–physical relationship between the measurements and the unknown parameters. This relationship is sufficiently well known for most applications. With regard to the stochastic model, two problem domains of fundamental importance arise: 1. How can stochastic models be set up as realistically as possible for the various geodetic observation methods and sensor systems? 2. How can the stochastic information be adequately considered in appropriate least squares adjustment models? Further questions include the interpretation of the stochastic properties of the computed target values with regard to precision and reliability and the use of the results for the detection of outliers in the input data (measurements). In this Special Issue, current research results on these general questions are presented in ten peer-reviewed articles. The basic findings can be applied to all technical scientific fields where measurements are used for the determination of parameters to describe geometric or physical phenomena.