1 INTRODUCTION  
2 TRANSFORMER MODEL ARCHITECTURE AND PERFORMANCE BOTTLENECKS  
3 HARDWARE DESIGN  
4 MODEL OPTIMIZATION  
5 MAPPING TRANSFORMERS TO HARDWARE  
6 ADAPTING TRANSFORMER ARCHITECTURE WITH NAS  
7 CONCLUSION