I. Introduction  

II. Preliminaries  

III. Problem Formulation  

IV. Benefits and Challenges of RL in Non-Stationary Environments  

V. Current Solution Approaches  

VI. Related Areas  

VII. Application Domains  

VIII. Future Directions  