Читать книгу Multi-Objective Decision Making - Diederik M. Roijers - Страница 7

Contents

Preface

Acknowledgments

Table of Abbreviations

1 Introduction

1.1 Motivation

1.2 Utility-based Approach

2 Multi-Objective Decision Problems

2.1 Multiple Objectives

2.2 Multi-Objective Coordination

2.2.1 Single-Objective Coordination Graphs

2.2.2 Multi-Objective Coordination Graphs

2.3 Multi-Objective Markov Decision Processes

2.3.1 Single-Objective Markov Decision Processes

2.3.2 Multi-Objective Markov Decision Processes

3 Taxonomy

3.1 Critical Factors

3.1.1 Single vs. Multiple Policies

3.1.2 Linear vs. Monotonically Increasing Scalarization Functions

3.1.3 Deterministic vs. Stochastic Policies

3.2 Solution Concepts

3.2.1 Case #1: Linear Scalarization and a Single Policy

3.2.2 Case #2: Linear Scalarization and Multiple Policies

3.2.3 Case #3: Monotonically Increasing Scalarization and a Single Deterministic Policy

3.2.4 Case #4: Monotonically Increasing Scalarization and a Single Stochastic Policy

3.2.5 Case #5: Monotonically Increasing Scalarization and Multiple Deterministic Policies

3.2.6 Case #6: Monotonically Increasing Scalarization and Multiple Stochastic Policies

3.3 Implications for MO-CoGs

3.4 Approximate Solution Concepts

3.5 Beyond the Taxonomy

4 Inner Loop Planning

4.1 Inner Loop Approach

4.1.1 A Simple MO-CoG

4.1.2 Finding a PCS

4.1.3 Finding a CCS

4.1.4 Design Considerations

4.2 Inner Loop Planning for MO-CoGs

4.2.1 Variable Elimination

4.2.2 Transforming the MO-CoG

4.2.3 Multi-Objective Variable Elimination

4.2.4 Comparing PMOVE and CMOVE

4.3 Inner Loop Planning for MOMDPs

4.3.1 Value Iteration

4.3.2 Multi-Objective Value Iteration

4.3.3 Pareto vs. Convex Value Iteration

5 Outer Loop Planning

5.1 Outer Loop Approach

5.2 Scalarized Value Functions

5.2.1 The Relationship with POMDPs

5.3 Optimistic Linear Support

5.4 Analysis

5.5 Approximate Single-Objective Solvers

5.6 Value Reuse

5.7 Comparing an Inner and Outer Loop Method

5.7.1 Theoretical Comparison

5.7.2 Empirical Comparison

5.8 Outer Loop Methods for PCS Planning

6 Learning

6.1 Offline MORL

6.2 Online MORL

7 Applications

7.1 Energy

7.2 Health

7.3 Infrastructure and Transportation

8 Conclusions and Future Work

8.1 Conclusions

8.2 Future Work

8.2.1 Scalarization of Expectation vs. Expectation of Scalarization

8.2.2 Other Decision Problems

8.2.3 Users in the Loop

Bibliography

Authors’ Biographies