1. Introduction

2. Problem Formulations and Related Theory

3. Online Supervised Learning

4. Applied Online Learning for Supervised Learning

5. Bandit Online Learning

6. Online Active Learning

7. Online Semi-supervised Learning

8. Online Unsupervised Learning

9. Related Areas and Other Terminologies

10. Conclusions