1 Introduction

2 Theoretical Background

3 Modalities of Interaction

4 Behavior Representations Learned from Interactions

6 Model Representations (Function Approximation)

7 On/Off Policy Learning

8 Reinforcement Learning with Human-in-the-Loop

9 Interfaces

10 User Studies in IIL

11 Benchmarks and Applications

12 Research Challenges and Opportunities

13 Conclusion