trillionlabs/Zoro

English | 한국어

Zoro is a two-stream block-diffusion language model (research checkpoint). It is produced by continued SFT with the full-mask recipe — 100% of each block masked (single view, no complementary mask) at block size bd=8, fresh optimizer, LR 5e-5, ~2k steps — starting from the base SFT checkpoint (v6, step_18000).

Code

Training and serving code lives in trillion-labs/trida-stack — the two-stream block-diffusion training stack (train/) and the HybridDiffusion SGLang serving fork (inference/). See the getting-started notebooks there for first-run training and inference.

How to serve

Best served self-speculative (draft with the diffusion stream, verify with the clean/AR stream): verify-guaranteed AR quality, and — because full-mask training shapes exactly the single-shot full-block prediction a self-spec draft needs — a higher acceptance length than the base, so it's the fastest self-spec at low concurrency. It also serves in plain AR/causal mode. Pure iterative diffusion decoding is not supported by this recipe (the shift decoder's partial-mask states are untrained). Runs on the HybridDiffusion SGLang fork. mask_id = 248077, vocab 248078.

Accuracy

Concurrency 1, threshold 0.9, reasoning on. A vanilla Qwen3.5-4B baseline is being added.

benchmark	Zoro · self-spec	Zoro · AR
GSM8K	84.2	89.3
MMLU-Pro	63.0	—
IFEval (prompt-strict)	69.7	—
BFCL (Non-Live AST)	65.2	—
Ko-AgentBench L1 / L2	0.909 / 1.00	—
FunctionChat call / compl	0.85 / 0.88	—
tau2 retail / air / tel	0.40 / 0.50 / 0.80	—

Serving speed & parallel commit (concurrency 1)

Block-diffusion self-spec commits multiple tokens per forward pass at verify-guaranteed AR quality. Zoro's full-mask training makes it a better drafter, so it accepts more per step and runs faster than the base.