1. Introduction

2. Notation and Background

3. Categorization of Exploratory Techniques

4. Reward-Free Exploration

5. Randomized Action Selection

6. Bonus-Based/Optimism-Based Exploration

7. Deliberate Exploration

8. Probability Matching

9. Conclusion and Perspectives