suproteem/StoriesLM-v2-1902

StoriesLM: A Family of Language Models With Time-Indexed Training Data

StoriesLM-v2 is a family of 125 encoder-only language models trained on an expanding sequence of historical language data.

More details are available at suproteem.is/writing/storieslm.

Usage

from transformers import AutoTokenizer, AutoModelForMaskedLM

model_id = "suproteem/StoriesLM-v2-1902"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)

from transformers import pipeline

fill_mask = pipeline("fill-mask", model="suproteem/StoriesLM-v2-1902")
fill_mask("We walked back and [MASK].")

Citation

@article{sarkar2024storieslm,
  author = {Sarkar, Suproteem},
  title = {StoriesLM: A Family of Language Models With Time-Indexed Training Data},
  journal = {SSRN Electronic Journal},
  year = {2024},
  url = {https://ssrn.com/abstract=4881024}
}