Multi-Objective Decision Making

Multi-Objective Decision Making

Diederik Roijers, Shimon Whiteson
ISBN: 9781627059602 | PDF ISBN: 9781627056991
Copyright © 2017 | 129 Pages | Publication Date: April, 2017

BEFORE YOU ORDER: You may have Academic or Corporate access to this title. Click here to find out: 10.2200/S00765ED1V01Y201704AIM034


Ordering Options: Paperback $39.95   E-book $31.96   Paperback & E-book Combo $49.94

Why pay full price? Members receive 15% off all orders.
Learn More Here

Read Our Digital Content License Agreement (pop-up)

Purchasing Options:

Many real-world decision problems have multiple objectives. For example, when choosing a medical treatment plan, we want to maximize the efficacy of the treatment, but also minimize the side effects. These objectives typically conflict, e.g., we can often increase the efficacy of the treatment, but at the cost of more severe side effects. In this book, we outline how to deal with multiple objectives in decision-theoretic planning and reinforcement learning algorithms. To illustrate this, we employ the popular problem classes of multi-objective Markov decision processes (MOMDPs) and multi-objective coordination graphs (MO-CoGs).

First, we discuss different use cases for multi-objective decision making, and why they often necessitate explicitly multi-objective algorithms. We advocate a utility-based approach to multi-objective decision making, i.e., that what constitutes an optimal solution to a multi-objective decision problem should be derived from the available information about user utility. We show how different assumptions about user utility and what types of policies are allowed lead to different solution concepts, which we outline in a taxonomy of multi-objective decision problems.

Second, we show how to create new methods for multi-objective decision making using existing single-objective methods as a basis. Focusing on planning, we describe two ways to creating multi-objective algorithms: in the inner loop approach, the inner workings of a single-objective method are adapted to work with multi-objective solution concepts; in the outer loop approach, a wrapper is created around a single-objective method that solves the multi-objective problem as a series of single-objective problems. After discussing the creation of such methods for the planning setting, we discuss how these approaches apply to the learning setting.

Next, we discuss three promising application domains for multi-objective decision making algorithms: energy, health, and infrastructure and transportation. Finally, we conclude by outlining important open problems and promising future directions.

Table of Contents

Table of Abbreviations
Multi-Objective Decision Problems
Inner Loop Planning
Outer Loop Planning
Conclusions and Future Work
Authors' Biographies

About the Author(s)

Diederik Roijers, University of Oxford
Diederik M. Roijers completed his master's in Computing Science at Utrecht University before obtaining his Ph.D. in Artificial Intelligence under the supervision of Shimon Whiteson and Frans A. Oliehoek at the University of Amsterdam in 2016. He then joined the University of Oxford as a postdoctoral research assistant. He was awarded a Postdoctoral Fellowship Grant from the FWO (Research Foundation - Flanders) and started as an FWO Postdoctoral Fellow at the Vrije Universiteit Brussel in October 2016. His research focuses on creating intelligent autonomous systems that assist humans in solving complex problems, especially those with multiple objectives. To this end, he focuses ondecision-theoretic planning and learning, which enable agents to use mathematical models to reason about the environments in which they operate. In the multi-objective problems he has been studying, the agents produce a set of possibly optimal policies that offer different trade-offs with respect to the objectives, to help users make an informed decision.

Shimon Whiteson, University of Oxford
Shimon Whiteson studied English and Computer Science at Rice University before completing his doctorate in Computer Science under the supervision of Peter Stone at the University of Texas at Austin in 2007. He then spent eight years as an Assistant and then an Associate Professor at the University of Amsterdam before joining the University of Oxford as an Associate Professor in 2015. He was awarded an ERC Starting Grant from the European Research Council in 2014. His research focuses on artificial intelligence with the goal of designing, analyzing, and evaluating the algorithms that enable computational systems to acquire and execute intelligent behavior. He is particularly interested in machine learning, with which computers can learn from experience, and decision-theoretic planning, with which they can reason about their goals and deduce behavioral strategies that maximize their utility. In addition to theoretical work on these topics, he has in recent years also focused on applying them to practical problems in robotics and search engine optimization.


Browse by Subject
Case Studies in Engineering
ACM Books
SEM Books
0 items

Note: Registered customers go to: Your Account to subscribe.

E-Mail Address:

Your Name: