1. Introduction  
2. Problem Formulation  
3. Classification  
4. Model-free Methods  
5. Model-based Offline Methods  
6. Theoretical Guarantees  
7. Realtime Methods  
8. Instance-specific Methods  
9. Multi-objective Methods  
10. Dynamic Methods  
11. Research Directions  
12. Conclusion