1 Introduction  
2 Related Surveys  
3 Human-in-the-Loop NLP Tasks  
4 Human-in-the-Loop Goals  
5 Human-machine Interaction  
6 How to Use User Feedback  
7 Research Directions & Open Problems  
8 Conclusion