最大token数

#24

by PatrickStar2 - opened Jun 21, 2024

max_seq_length 是512，但tokenizer_config.json里面的model_max_length参数为1000000000000000019884624838656。实际用时，不能超过512，那么model_max_length这个参数是什么意思？应该如何用？

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment