1 Introduction

2 Preliminary and Taxonomy

3 LLM Architecture Design

4 LLM Pre-training

5 LLM Fine-tuning

6 LLM Inference

7 System Design

8 Technique Categorization by Resources

9 Benchmark and Evaluation Metrics

10 Open Challenges and Future Directions

11 Conclusion