General Theory of Markov Processes

This graduate-level text and reference in probability, with numerous applications to several fields of science, presents nonmeasure-theoretic introduction to theory of Markov processes. The work also covers mathematical models based on the theory, employed in various applied fields. Prerequisites are a knowledge of elementary probability theory, mathematical statistics, and analysis. Appendixes. Bibliographies. 1960 edition.

From the Reviews: "To call this work encyclopedic would not give an accurate picture of its content and style. Some parts read like a textbook, but others are more technical and contain relatively new results. ... The exposition is robust and explicit, as one has come to expect of the Russian tradition of mathematical writing." --K.L. Chung, American Scientist, 1977

The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South American and Asian scholars.

This clear presentation of themost fundamental models ofrandom phenomena employsmethods that recognize computerrelatedaspects of theory. Topicsinclude probability spaces andrandom variables, expectationsand independence, Bernoulliprocesses and sums of independentrandom variables, Poisson processes, Markov chainsand processes, and renewal theory. Assuming only a backgroundin calculus, this outstanding text includes an introductionto basic stochastic processes.Reprint of the Prentice-Hall Publishers, Englewood Cliffs,New Jersey, 1975 edition.

Over the past eighty years, martingales have become central in the mathematics of randomness. They appear in the general theory of stochastic processes, in the algorithmic theory of randomness, and in some branches of mathematical statistics. Yet little has been written about the history of this evolution. This book explores some of the territory that the history of the concept of martingales has transformed. The historian of martingales faces an immense task. We can find traces of martingale thinking at the very beginning of probability theory, because this theory was related to gambling, and the evolution of a gambler’s holdings as a result of following a particular strategy can always be understood as a martingale. More recently, in the second half of the twentieth century, martingales became important in the theory of stochastic processes at the very same time that stochastic processes were becoming increasingly important in probability, statistics and more generally in various applied situations. Moreover, a history of martingales, like a history of any other branch of mathematics, must go far beyond an account of mathematical ideas and techniques. It must explore the context in which the evolution of ideas took place: the broader intellectual milieux of the actors, the networks that already existed or were created by the research, even the social and political conditions that favored or hampered the circulation and adoption of certain ideas. This books presents a stroll through this history, in part a guided tour, in part a random walk. First, historical studies on the period from 1920 to 1950 are presented, when martingales emerged as a distinct mathematical concept. Then insights on the period from 1950 into the 1980s are offered, when the concept showed its value in stochastic processes, mathematical statistics, algorithmic randomness and various applications.

An introduction to general theories of stochastic processes and modern martingale theory. The volume focuses on consistency, stability and contractivity under geometric invariance in numerical analysis, and discusses problems related to implementation, simulation, variable step size algorithms, and random number generation.

This textbook explores two distinct stochastic processes that evolve at random: weakly stationary processes and discrete parameter Markov processes. Building from simple examples, the authors focus on developing context and intuition before formalizing the theory of each topic. This inviting approach illuminates the key ideas and computations in the proofs, forming an ideal basis for further study. After recapping the essentials from Fourier analysis, the book begins with an introduction to the spectral representation of a stationary process. Topics in ergodic theory follow, including Birkhoff’s Ergodic Theorem and an introduction to dynamical systems. From here, the Markov property is assumed and the theory of discrete parameter Markov processes is explored on a general state space. Chapters cover a variety of topics, including birth–death chains, hitting probabilities and absorption, the representation of Markov processes as iterates of random maps, and large deviation theory for Markov processes. A chapter on geometric rates of convergence to equilibrium includes a splitting condition that captures the recurrence structure of certain iterated maps in a novel way. A selection of special topics concludes the book, including applications of large deviation theory, the FKG inequalities, coupling methods, and the Kalman filter. Featuring many short chapters and a modular design, this textbook offers an in-depth study of stationary and discrete-time Markov processes. Students and instructors alike will appreciate the accessible, example-driven approach and engaging exercises throughout. A single, graduate-level course in probability is assumed.

The general topic of this book is the ergodic behavior of Markov processes. A detailed introduction to methods for proving ergodicity and upper bounds for ergodic rates is presented in the first part of the book, with the focus put on weak ergodic rates, typical for Markov systems with complicated structure. The second part is devoted to the application of these methods to limit theorems for functionals of Markov processes. The book is aimed at a wide audience with a background in probability and measure theory. Some knowledge of stochastic processes and stochastic differential equations helps in a deeper understanding of specific examples. Contents Part I: Ergodic Rates for Markov Chains and Processes Markov Chains with Discrete State Spaces General Markov Chains: Ergodicity in Total Variation MarkovProcesseswithContinuousTime Weak Ergodic Rates Part II: Limit Theorems The Law of Large Numbers and the Central Limit Theorem Functional Limit Theorems

This unique volume presents a collection of the extensive journal publications written by Kai Lai Chung over a span of 70-odd years. It was produced to celebrate his 90th birthday. The selection is only a subset of the many contributions that he made throughout his prolific career. Another volume, Chance and Choice, published by World Scientific in 2004, contains yet another subset, with four articles in common with this volume. Kai Lai Chung''s research contributions have had a major influence on several areas in probability. Among his most significant works are those related to sums of independent random variables, Markov chains, time reversal of Markov processes, probabilistic potential theory, Brownian excursions, and gauge theorems for the SchrAdinger equation.As Kai Lai Chung''s contributions spawned critical new developments, this volume also contains retrospective and perspective views provided by collaborators and other authors who themselves advanced the areas of probability and mathematics."