Markov Decision Processes (MDPs) are a mathematical framework for modeling sequential decision problems under uncertainty as well as reinforcement learning problems. Written by experts in the field, this book provides a global view of current research using MDPs in artificial intelligence. It starts with an introductory presentation of the fundamental aspects of MDPs (planning in MDPs, reinforcement learning, partially observable MDPs, Markov games and the use of non-classical criteria). It then presents more advanced research trends in the field and gives some concrete examples using illustrative real life applications.

Markov Decision Processes (MDPs) are widely popular in Artificial Intelligence for modeling sequential decision-making scenarios with probabilistic dynamics. They are the framework of choice when designing an intelligent agent that needs to act for long periods of time in an environment where its actions could have uncertain outcomes. MDPs are actively researched in two related subareas of AI, probabilistic planning and reinforcement learning. Probabilistic planning assumes known models for the agent's goals and domain dynamics, and focuses on determining how the agent should behave to achieve its objectives. On the other hand, reinforcement learning additionally learns these models based on the feedback the agent gets from the environment. This book provides a concise introduction to the use of MDPs for solving probabilistic planning problems, with an emphasis on the algorithmic perspective. It covers the whole spectrum of the field, from the basics to state-of-the-art optimal and approximation algorithms. We first describe the theoretical foundations of MDPs and the fundamental solution techniques for them. We then discuss modern optimal algorithms based on heuristic search and the use of structured representations. A major focus of the book is on the numerous approximation schemes for MDPs that have been developed in the AI literature. These include determinization-based approaches, sampling techniques, heuristic functions, dimensionality reduction, and hierarchical representations. Finally, we briefly introduce several extensions of the standard MDP classes that model and solve even more complex planning problems. Table of Contents: Introduction / MDPs / Fundamental Algorithms / Heuristic Search Algorithms / Symbolic Algorithms / Approximation Algorithms / Advanced Notes

Provides a comprehensive survey of techniques to automatically construct basis functions or features for value function approximation in Markov decision processes and reinforcement learning.

This book constitutes the proceedings of the 38th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, AI 2018, held in Cambridge, UK, in December 2018. The 25 full papers and 12 short papers presented in this volume were carefully reviewed and selected from 46 submissions. There are technical and application papers which were organized in topical sections named: Neural Networks; Planning and Scheduling; Machine Learning; Industrial Applications of Artificial Intelligence; Planning and Scheduling in Action; Machine Learning in Action; Applications of Machine Learning; and Applications of Agent Systems and Genetic Algorithms.

Decision making arises when we wish to select the best possible course of action from a set of alternatives. With advancements of the digital technologies, it is easy, and almost instantaneous, to gather a large volume of information and/or data pertaining to a problem that we want to solve. For instance, the world-wi- web is perhaps the primary source of information and/or data that we often turn to when we face a decision making problem. However, the information and/or data that we obtain from the real world often are complex, and comprise various kinds of noise. Besides, real-world information and/or data often are incomplete and ambiguous, owing to uncertainties of the environments. All these make decision making a challenging task. To cope with the challenges of decision making, - searchers have designed and developed a variety of decision support systems to provide assistance in human decision making processes. The main aim of this book is to provide a small collection of techniques stemmed from artificial intelligence, as well as other complementary methodo- gies, that are useful for the design and development of intelligent decision support systems. Application examples of how these intelligent decision support systems can be utilized to help tackle a variety of real-world problems in different - mains, e. g. business, management, manufacturing, transportation and food ind- tries, and biomedicine, are also presented. A total of twenty chapters, which can be broadly divided into two parts, i. e.

Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive decision making in multi-agent social learning where there is herding and data incest? And how can sophisticated radars and sensors adapt their sensing in real time?

This book constitutes the refereed proceedings of the Third International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2006, held in Tarragona, Spain, in April 2006. The 31 revised full papers presented together with 4 invited lectures were thoroughly reviewed and selected from 97 submissions. The papers are devoted to theory and tools for modeling decisions, as well as applications that encompass decision making processes and information fusion techniques.

Eugene A. Feinberg Adam Shwartz This volume deals with the theory of Markov Decision Processes (MDPs) and their applications. Each chapter was written by a leading expert in the re spective area. The papers cover major research areas and methodologies, and discuss open questions and future research directions. The papers can be read independently, with the basic notation and concepts ofSection 1.2. Most chap ters should be accessible by graduate or advanced undergraduate students in fields of operations research, electrical engineering, and computer science. 1.1 AN OVERVIEW OF MARKOV DECISION PROCESSES The theory of Markov Decision Processes-also known under several other names including sequential stochastic optimization, discrete-time stochastic control, and stochastic dynamic programming-studiessequential optimization ofdiscrete time stochastic systems. The basic object is a discrete-time stochas tic system whose transition mechanism can be controlled over time. Each control policy defines the stochastic process and values of objective functions associated with this process. The goal is to select a "good" control policy. In real life, decisions that humans and computers make on all levels usually have two types ofimpacts: (i) they cost orsavetime, money, or other resources, or they bring revenues, as well as (ii) they have an impact on the future, by influencing the dynamics. In many situations, decisions with the largest immediate profit may not be good in view offuture events. MDPs model this paradigm and provide results on the structure and existence of good policies and on methods for their calculation.

The International Conference on Intelligent Computing (ICIC) was formed to provide an annual forum dedicated to the emerging and challenging topics in artificial intelligence, machine learning, bioinformatics, and computational biology, etc. It aims to bring - gether researchers and practitioners from both academia and industry to share ideas, problems, and solutions related to the multifaceted aspects of intelligent computing. ICIC 2009, held in Ulsan, Korea, September 16-19, 2009, constituted the 5th - ternational Conference on Intelligent Computing. It built upon the success of ICIC 2008, ICIC 2007, ICIC 2006, and ICIC 2005 held in Shanghai, Qingdao, Kunming, and Hefei, China, 2008, 2007, 2006, and 2005, respectively. This year, the conference concentrated mainly on the theories and methodologies as well as the emerging applications of intelligent computing. Its aim was to unify the p- ture of contemporary intelligent computing techniques as an integral concept that hi- lights the trends in advanced computational intelligence and bridges theoretical research with applications. Therefore, the theme for this conference was “Emerging Intelligent Computing Technology and Applications.” Papers focusing on this theme were solicited, addressing theories, methodologies, and applications in science and technology.

This book provides an overview of the main methods and results in the formal study of the human decision-making process, as defined in a relatively wide sense. A key aim of the approach contained here is to try to break down barriers between various disciplines encompassed by this field, including psychology, economics and computer science. All these approaches have contributed to progress in this very important and much-studied topic in the past, but none have proved sufficient so far to define a complete understanding of the highly complex processes and outcomes. This book provides the reader with state-of-the-art coverage of the field, essentially forming a roadmap to the field of decision analysis. The first part of the book is devoted to basic concepts and techniques for representing and solving decision problems, ranging from operational research to artificial intelligence. Later chapters provide an extensive overview of the decision-making process under conditions of risk and uncertainty. Finally, there are chapters covering various approaches to multi-criteria decision-making. Each chapter is written by experts in the topic concerned, and contains an extensive bibliography for further reading and reference.