32k-compatible
#3
by exoplanet - opened
Hi awesome bakers, great job with this (v2) reranker family. On your technical blog, "32k-compatible" is mentioned, but I couldn't figure out how this works in practice. Any pointers would be appreciated. Krispy regards.
hey @exoplanet , we've fine-tuned the model up to a context length of 8k but it generally supports 32k.
You can just pass max_length to the model during initialisation:
reranker = MxbaiRerankV2(..., max_length=32768)
juliuslipp changed discussion status to closed