A path-breaking account of Markov decision processes-theory and computation This book's clear presentation of theory, numerous chapter-end problems, and development of a unified method for the computation of optimal policies in both discrete and continuous time make it an excellent course text for graduate students and advanced undergraduates. Its comprehensive coverage of important recent advances in stochastic dynamic programming makes it a valuable working resource for operations research professionals, management scientists, engineers, and others. Stochastic Dynamic Programming and the Control of Queueing Systems presents the theory of optimization under the finite horizon, infinite horizon discounted, and average cost criteria. It then shows how optimal rules of operation (policies) for each criterion may be numerically determined. A great wealth of examples from the application area of the control of queueing systems is presented. Nine numerical programs for the computation of optimal policies are fully explicated. The Pascal source code for the programs is available for viewing and downloading on the Wiley Web site at www.wiley.com/products/subject/mathematics. The site contains a link to the author's own Web site and is also a place where readers may discuss developments on the programs or other aspects of the material. The source files are also available via ftp at ftp://ftp.wiley.com/public/sci_tech_med/stochastic Stochastic Dynamic Programming and the Control of Queueing Systems features: * Path-breaking advances in Markov decision process techniques, brought together for the first time in book form * A theorem/proof format (proofs may be omitted without loss of continuity) * Development of a unified method for the computation of optimal rules of system operation * Numerous examples drawn mainly from the control of queueing systems * Detailed discussions of nine numerical programs * Helpful chapter-end problems * Appendices with complete treatment of background material

This book is dedicated to Jinhua Cao on the occasion of his 80th birthday. Jinhua Cao is one of the most famous reliability theorists. His main contributions include: published over 100 influential scientific papers; published an interesting reliability book in Chinese in 1986, which has greatly influenced the reliability of education, academic research and engineering applications in China; initiated and organized Reliability Professional Society of China (the first part of Operations Research Society of China) since 1981. The high admiration that Professor Cao enjoys in the reliability community all over the world was witnessed by the enthusiastic response of each contributor in this book. The contributors are leading researchers with diverse research perspectives. The research areas of the book iclude a broad range of topics related to reliability models, queueing theory, manufacturing systems, supply chain finance, risk management, Markov decision processes, blockchain and so forth. The book consists of a brief Preface describing the main achievements of Professor Cao; followed by congratulations from Professors Way Kuo and Wei Wayne Li, and by Operations Research Society of China, and Reliability Professional Society of China; and further followed by 25 articles roughly grouped together. Most of the articles are written in a style understandable to a wide audience. This book is useful to anyone interested in recent developments in reliability, network security, system safety, and their stochastic modeling and analysis.

This volume provides a general overview of discrete- and continuous-time Markov control processes and stochastic games, along with a look at the range of applications of stochastic control and some of its recent theoretical developments. These topics include various aspects of dynamic programming, approximation algorithms, and infinite-dimensional linear programming. In all, the work comprises 18 carefully selected papers written by experts in their respective fields. Optimization, Control, and Applications of Stochastic Systems will be a valuable resource for all practitioners, researchers, and professionals in applied mathematics and operations research who work in the areas of stochastic control, mathematical finance, queueing theory, and inventory systems. It may also serve as a supplemental text for graduate courses in optimal control and dynamic games.

Optimal Control and Optimization of Stochastic Supply Chain Systems examines its subject the context of the presence of a variety of uncertainties. Numerous examples with intuitive illustrations and tables are provided, to demonstrate the structural characteristics of the optimal control policies in various stochastic supply chains and to show how to make use of these characteristics to construct easy-to-operate sub-optimal policies. In Part I, a general introduction to stochastic supply chain systems is provided. Analytical models for various stochastic supply chain systems are formulated and analysed in Part II. In Part III the structural knowledge of the optimal control policies obtained in Part II is utilized to construct easy-to-operate sub-optimal control policies for various stochastic supply chain systems accordingly. Finally, Part IV discusses the optimisation of threshold-type control policies and their robustness. A key feature of the book is its tying together of the complex analytical models produced by the requirements of operational practice, and the simple solutions needed for implementation. The analytical models and theoretical analysis propounded in this monograph will be of benefit to academic researchers and graduate students looking at logistics and supply chain management from standpoints in operations research or industrial, manufacturing, or control engineering. The practical tools and solutions and the qualitative insights into the ideas underlying functional supply chain systems will be of similar use to readers from more industrially-based backgrounds.

This book deals with dynamic and stochastic methods for multi-project planning. Based on the idea of using queueing networks for the analysis of dynamic-stochastic multi-project environments this book addresses two problems: detailed scheduling of project activities, and integrated order acceptance and capacity planning. In an extensive simulation study, the book thoroughly investigates existing scheduling policies. To obtain optimal and near optimal scheduling policies new models and algorithms are proposed based on the theory of Markov decision processes and Approximate Dynamic programming. Then the book presents a new model for the effective computation of optimal policies based on a Markov decision process. Finally, the book provides insights into the structure of optimal policies.

This book constitutes the refereed proceedings of the 17th International Conference on Distributed Computer and Communication Networks, DCCN 2013, held in Moscow, Russia, in October 2013. The 22 revised full papers presented were carefully reviewed and selected from numerous submissions. The papers cover the following subjects: computer and communication networks architecture optimization; control in computer and communication networks; performance and QoS evaluation in wireless networks; modeling and simulation of network protocols; queueing theory; wireless IEEE 802.11, IEEE 802.15, IEEE 802.16 and UMTS (LTE) net-works; RFID technology and its application in intellectual transportation networks; protocols design (MAC, Routing) for centimeter and millimeter wave mesh networks; internet and web applications and services; application integration in distributed information systems.

This book constitutes the refereed proceedings of the 15th International Scientific Conference on Information Technologies and Mathematical Modeling, named after A. F. Terpugov, ITMM 2016, held in Katun, Russia, in September 2016. The 33 full papers presented together with 4 short papers were carefully reviewed and selected from 96 submissions. They are devoted to new results in the queueing theory and its applications, addressing specialists in probability theory, random processes, mathematical modeling as well as engineers dealing with logical and technical design and operational management of telecommunication and computer networks.

From foundations to state-of-the-art; the tools and philosophy you need to build network models.

Eugene A. Feinberg Adam Shwartz This volume deals with the theory of Markov Decision Processes (MDPs) and their applications. Each chapter was written by a leading expert in the re spective area. The papers cover major research areas and methodologies, and discuss open questions and future research directions. The papers can be read independently, with the basic notation and concepts ofSection 1.2. Most chap ters should be accessible by graduate or advanced undergraduate students in fields of operations research, electrical engineering, and computer science. 1.1 AN OVERVIEW OF MARKOV DECISION PROCESSES The theory of Markov Decision Processes-also known under several other names including sequential stochastic optimization, discrete-time stochastic control, and stochastic dynamic programming-studiessequential optimization ofdiscrete time stochastic systems. The basic object is a discrete-time stochas tic system whose transition mechanism can be controlled over time. Each control policy defines the stochastic process and values of objective functions associated with this process. The goal is to select a "good" control policy. In real life, decisions that humans and computers make on all levels usually have two types ofimpacts: (i) they cost orsavetime, money, or other resources, or they bring revenues, as well as (ii) they have an impact on the future, by influencing the dynamics. In many situations, decisions with the largest immediate profit may not be good in view offuture events. MDPs model this paradigm and provide results on the structure and existence of good policies and on methods for their calculation.

The aim of this Special Issue of Mathematics is to commemorate the outstanding Russian mathematician Vladimir Zolotarev, whose 90th birthday will be celebrated on February 27th, 2021. The present Special Issue contains a collection of new papers by participants in sessions of the International Seminar on Stability Problems for Stochastic Models founded by Zolotarev. Along with research in probability distributions theory, limit theorems of probability theory, stochastic processes, mathematical statistics, and queuing theory, this collection contains papers dealing with applications of stochastic models in modeling of pension schemes, modeling of extreme precipitation, construction of statistical indicators of scientific publication importance, and other fields.