1 INTRODUCTION  

2 PRELIMINARIES  

3 GRADIENT DESCENT OPTIMIZATION ALGORITHMS  

4 LARGE BATCH TRAINING  

5 GENERALIZATION GAP  

6 SECOND ORDER OPTIMIZATION  

7 COMMUNICATION  

8 MEMORY  

9 CONCLUSION