From foundations to state-of-the-art; the tools and philosophy you need to build network models.

The goal of this book is to provide an approach to the corresponding geometric theory of nonautonomous discrete dynamical systems in infinite-dimensional spaces by virtue of 2-parameter semigroups (processes).

Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive decision making in multi-agent social learning where there is herding and data incest? And how can sophisticated radars and sensors adapt their sensing in real time?

This fundamental exposition of queueing theory, written by leading researchers, answers the need for a mathematically sound reference work on the subject and has become the standard reference. The thoroughly revised second edition contains a substantial number of exercises and their solutions, which makes the book suitable as a textbook.

Stochastic Geometry is the mathematical discipline which studies mathematical models for random geometric structures. This book collects lectures presented at the CIME summer school in Martina Franca in September 2004. The main lecturers covered Spatial Statistics, Random Points, Integral Geometry and Random Sets. These are complemented by two additional contributions on Random Mosaics and Crystallization Processes. The book presents a comprehensive and up-to-date description of important aspects of Stochastic Geometry.

Monotonicity in Markov Reward and Decision Chains: Theory and Applications focuses on monotonicity results for dynamic systems that take values in the natural numbers or in more-dimensional lattices. The results are mostly formulated in terms of controlled queueing systems, but there are also applications to maintenance systems, revenue management, and so forth. The focus is on results that are obtained by inductively proving properties of the dynamic programming value function. A framework is provided for using this method that unifies results obtained for different models. The author also provides a comprehensive overview of the results that can be obtained through it, in which he discusses not only (partial) characterizations of optimal policies but also applications of monotonicity to optimization problems and the comparison of systems. Monotonicity in Markov Reward and Decision Chains: Theory and Applications is an invaluable resource for anyone planning or conducting research in this particular area. The essentials of the topic are presented in an accessible manner and an extensive bibliography guides towards further reading.

These lectures concentrate on (nonlinear) stochastic partial differential equations (SPDE) of evolutionary type. There are three approaches to analyze SPDE: the "martingale measure approach", the "mild solution approach" and the "variational approach". The purpose of these notes is to give a concise and as self-contained as possible an introduction to the "variational approach". A large part of necessary background material is included in appendices.