LQK — BabyLM 2026 Strict
LQK is a 116M-parameter causal model built on a Llama-1-style decoder with Qwen3-Next-style gated attention and Kimi Attention Residuals (AttnRes). This seed-1337 model was trained for 10 epochs on the official BabyLM 2026 Strict data with its exact 16k byte-level BPE tokenizer.
Load with AutoModelForCausalLM.from_pretrained("ypwhere/LQK-BabyLM-Strict", trust_remote_code=True). The repository provides main plus the 28 required chck_*M checkpoint revisions. Training-data terms remain governed by the BabyLM release; the included nanoGPT-derived model code is provided for research reproducibility.
- Downloads last month
- 29