Acknowledge license to accept the repository

Access is granted automatically after you complete the form. State a concrete research field and topic.

You must agree to the Zlatorog Research Only License before downloading these weights.

Production deployment is forbidden even if unpaid: you may not serve this model as a product, SaaS, public-facing service, or internal production system, including university or non-profit production use.

Allowed: research, evaluation, benchmarking, and academic publication.

Not allowed: commercial use; production deployment (paid or unpaid); distributing derivatives except under the same research-only license.

This is not CC-BY-NC-ND. See LICENSE in the repository.

Describe a concrete research field and topic (or evaluation) you will use these weights for. Access is granted after you complete this form.

By requesting access you confirm that you have read LICENSE and will use the Materials for research and evaluation only.

Log in or Sign Up to review the conditions and access this model content.

Zlatorog-30B-A3B-Thinking-Experimental

Experimental Slovenian-first thinking checkpoint of Zlatorog 30B MoE. Intended to be used with chain-of-thought traces. Research and evaluation only. See USAGE.md for load and serve notes.

This is not the dense 12B Zlatorog line (Zlatorog-12B-Instruct-Beta). Collection: zID4si/zlatorog-moe-experimental.

House freeze alias (not part of the Hub name): zlatorog-max-sft-v2 (HF merge). LIMA names a process (capped CPT/SFT data), not this weight file.

What this is

Zlatorog continues Qwen3-30B-A3B as a Mixture-of-Experts language model (about 3B parameters active per token). Continual pretraining used a LIMA-inspired data diet: a capped token budget and content filtering of Slovenian web text. Filtering draws on zID4si/CLASSLA_web_sl_2_enriched and the educational-value scorer zID4si/slovenian-edu-classifier-slovlo.

The tokenizer adds South-Slavic vocabulary so Slovenian text is more token-efficient than the upstream 30B-A3B tokenizer. That expansion slightly grows embedding and output layers (on the order of 0.1B parameters). The checkpoint is still in the 30.6B-total / 3B-active ballpark; routing width is unchanged.

Supervised fine-tuning used a LIMA-style capped mix with thinking traces in the labels. Mix sizes belong in a paper, not on this card.

trust_remote_code=True is tokenizer-only. The architecture is stock Qwen3MoeForCausalLM. The bundled tokenizer matches zidsi/Zlatorog-30B-MoE-tokenizer.

Observed behaviour (qualitative)

Strengths we observed in-house: Slovenian multi-step math and similar tasks are where the trace helps; this SKU is meant to run with thinking on.

Limits: thinking traces in this release can fill or exceed the native 32,768-token context, so the visible final answer can be empty or truncated if you do not cap generation. Quality is uneven if thinking is left off. English open generation is not the SKU’s job. Native context is 32,768 tokens (default RoPE). Optional longer windows are unevaluated (see USAGE Advanced serving).

Intended use

Research, evaluation, and academic publication under the Zlatorog Research Only License. Production deployment is forbidden even if unpaid.

How to load

See USAGE.md. Default path: Transformers from_pretrained on this repo with trust_remote_code=True and thinking on. Cap max_new_tokens so the trace plus answer can finish inside 32k.

Citation

If you use Zlatorog-30B-A3B-Thinking-Experimental in your research, please cite it as follows:

BibTeX

@misc{zlatorog30bthinking2026,
  title        = {Zlatorog-30B-A3B-Thinking-Experimental: A Slovenian Thinking Mixture-of-Experts Language Model},
  author       = {Tomaž Savodnik, Zavod za informacijsko družbo (zID)},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/zID4si/Zlatorog-30B-A3B-Thinking-Experimental}},
  note         = {Version Experimental}
}

APA

Savodnik, T., Zavod za informacijsko družbo (zID). (2026). Zlatorog-30B-A3B-Thinking-Experimental: A Slovenian Thinking Mixture-of-Experts Language Model [Computer software]. Hugging Face. https://huggingface.co/zID4si/Zlatorog-30B-A3B-Thinking-Experimental

Downloads last month
-
Safetensors
Model size
31B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for zID4si/Zlatorog-30B-A3B-Thinking-Experimental

Finetuned
(90)
this model

Collection including zID4si/Zlatorog-30B-A3B-Thinking-Experimental