Planning with Markov Decision Processes

Planning with Markov Decision Processes

An AI Perspective

Mausam, Andrey Kolobov,
ISBN: 9781608458868 | PDF ISBN: 9781608458875
Copyright © 2012 | 210 Pages | Publication Date: 01/01/2012

BEFORE YOU ORDER: You may have Academic or Corporate access to this title. Click here to find out: 10.2200/S00426ED1V01Y201206AIM017

Ordering Options: Paperback $45.00   E-book $36.00   Paperback & E-book Combo $56.25

Why pay full price? Members receive 15% off all orders.
Learn More Here

Read Our Digital Content License Agreement (pop-up)

Purchasing Options:

Markov Decision Processes (MDPs) are widely popular in Artificial Intelligence for modeling sequential decision-making scenarios with probabilistic dynamics. They are the framework of choice when designing an intelligent agent that needs to act for long periods of time in an environment where its actions could have uncertain outcomes. MDPs are actively researched in two related subareas of AI, probabilistic planning and reinforcement learning. Probabilistic planning assumes known models for the agent's goals and domain dynamics, and focuses on determining how the agent should behave to achieve its objectives. On the other hand, reinforcement learning additionally learns these models based on the feedback the agent gets from the environment.

This book provides a concise introduction to the use of MDPs for solving probabilistic planning problems, with an emphasis on the algorithmic perspective. It covers the whole spectrum of the field, from the basics to state-of-the-art optimal and approximation algorithms. We first describe the theoretical foundations of MDPs and the fundamental solution techniques for them. We then discuss modern optimal algorithms based on heuristic search and the use of structured representations. A major focus of the book is on the numerous approximation schemes for MDPs that have been developed in the AI literature. These include determinization-based approaches, sampling techniques, heuristic functions, dimensionality reduction, and hierarchical representations. Finally, we briefly introduce several extensions of the standard MDP classes that model and solve even more complex planning problems.

Table of Contents

Fundamental Algorithms
Heuristic Search Algorithms
Symbolic Algorithms
Approximation Algorithms
Advanced Notes

About the Author(s)

Mausam, University of Washington
Mausam is a Research Assistant Professor at the Turing Center in the Department of Computer Science at the University of Washington, Seattle. His research interests span various sub-fields of artificial intelligence, including sequential decision making under uncertainty, large scale natural language processing, Web information systems, heuristic search, machine learning, and AI applications to crowd-sourcing. Mausam obtained a Ph.D. from University of Washington in 2007 and a Bachelor of Technology from IIT Delhi in 2001. His PhD thesis was awarded honorable mention for the 2008 ICAPS Best Dissertation Award. Mausam has written scores of papers in top AI conferences and journals. He has served on the senior program committees of AI conferences such as AAAI and IJCAI, program committees of several other conferences, and on NSF panels.

Andrey Kolobov, University of Washington
Andrey Kolobov is a Ph.D. student at the Department of Computer Science at the University of Washington, Seattle, advised by Professors Mausam and Daniel S. Weld. His research concentrates primarily on designing scalable domain-independent algorithms for planning under uncertainty, but has touched upon other areas as well,including information retrieval, robotics, first-order probabilistic languages, and transfer learning. Andrey has authored multiple works in top AI venues, including a publication in the exceptional paper track at AAAI-2010, the best paper at the Doctoral Consortium at ICAPS-2009, and a runner-up entry at the International Probabilistic Planning Competition - 2011. He is also a recipient of the Outstanding Program Committee Member Award at AAAI-2012.


Related Series

Human Language Technologies


Browse by Subject
Case Studies in Engineering
ACM Books
SEM Books
0 items

Note: Registered customers go to: Your Account to subscribe.

E-Mail Address:

Your Name: