Measured on device (edge-compat, spark-x2.5-1.7b-int4): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 17.8 tok/s · prefill 459 tok/s · TTFT 520 ms · all 1733 ops delegated (2026-09-07); Mac Studio M4 Max · LiteRT-LM 0.17.0 · GPU · decode 103.6 tok/s · prefill 2492 tok/s · TTFT 112 ms (2026-09-07); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 13.6 tok/s · prefill 83 tok/s · TTFT 2.65 s (2026-09-07). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/spark-x2.5-1.7b-int4/CARD.md

Measured on device (edge-compat, spark-x2.5-1.7b-int8): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 15.8 tok/s · prefill 637 tok/s · TTFT 400 ms · all 1736 ops delegated (2026-09-07); Mac Studio M4 Max · LiteRT-LM 0.17.0 · GPU · decode 95.8 tok/s · prefill 2308 tok/s · TTFT 121 ms (2026-09-07); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 12.3 tok/s · prefill 165 tok/s · TTFT 1.37 s (2026-09-07). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/spark-x2.5-1.7b-int8/CARD.md

Spark-X2.5-1.7B — LiteRT-LM

XHToken/Spark-X2.5-1.7B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. Tested on litert-lm 0.17.0.

Spark-X2.5 is the SparkLLM Team's (iFLYTEK) open model series released in August 2026 — a 1.71B dense decoder (28 layers, hidden 2048, 8 query / 2 KV heads of 256, sliding-window(512) x3 : full x1, 131,072-entry tied vocab) with a hybrid attention layout of three sliding-window layers per full-attention layer, per-head sigmoid attention-output gates, exact-GELU gated MLPs and a native 1M-token context in the original. It is a reasoning model: it works problems inside <think>…</think> before answering, and these bundles carry that machinery — the generation prompt pre-fills the think opener exactly as the vendor chat template does, and a thought channel is declared in the bundle metadata so the runtime separates reasoning from the answer and honours a thinking budget. Apache-2.0.

File	Recipe	Size
Spark-X2.5-1.7B_int8.litertlm	int8 dynamic on linears + embedding	1835 MB
Spark-X2.5-1.7B_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 embedding (externalized)	1263 MB

int8 is the recommended file — GSM8K at bf16 parity (76 = 76) and 8/8 on every gate. int4 (block-32) is the phone / size option at 1.26 GB: it costs 10 GSM8K points (66), about half of that from longer reasoning chains meeting the 3584-token budget rather than wrong reasoning; it decodes faster than int8 on the Mac GPU (103.6 vs 95.8 tok/s) and on the Galaxy S26 (GPU 17.4–17.8 vs 13.9–15.8, CPU 13.6–13.9 vs 12.0–12.3), and scored 8/8 on the iPhone 17 Pro on both backends. On the S26 the GPU is the path to use for either file: same-or-faster decode, 2× the CPU prefill, and less than half the CPU path's peak memory.

Correctness

• 8-question sanity gate (Apple M4 Max, litert-lm 0.17.0 CLI, one process per question, greedy, thinking on): int8 CPU 8/8 / GPU 8/8, int4 CPU 8/8 / GPU 8/8; the bf16 PyTorch reference scores 8/8 on the same questions through the vendor chat template. Every answer arrived as a clean final answer through the thought channel — no reasoning leakage, no degeneration, every think block closed.
• Tokenizer parity: the bundle's tokenizer section (the upstream tokenizer.json, byte-level BPE) encodes 234 of 234 probe rows — every added token alone and mid-string, the role-marker turn, Latin-1 / Extended-A, emoji, CJK, digits, whitespace — to the same ids as the tokenizers reading of the upstream file.
• Multi-turn (3 turns, python API, greedy: plant a fact, do arithmetic, recall the fact): pass on both files, with and without the runtime's channel filtering; no marker leakage.

Accuracy — GSM8K

GSM8K, greedy, 0-shot chain-of-thought, 3584 output tokens (a reasoning model truncated mid-thought never reaches its answer; 2048 cut a third of the bf16 model's first questions), n=100, identical prompt, harness and answer extraction for every row; the LiteRT rows ran on the litert-lm 0.17.0 engine (GPU backend), scored on the text after the thought channel:

Configuration	GSM8K
PyTorch bf16 (reference, MPS)	76% (20 unfinished at the 3584-token cap)
LiteRT int8	76% (20 unfinished at the 3584-token cap)
LiteRT int4	66% (27 unfinished at the 3584-token cap)

Usage

litert-lm run ./Spark-X2.5-1.7B_int8.litertlm --prompt "What is the capital of France? Answer in one word." --thinking true

# GPU
litert-lm run ./Spark-X2.5-1.7B_int8.litertlm --backend gpu --cache no --thinking true --prompt "..."

Notes for a reasoning model:

• Give it a generous output budget (≥ 2048 tokens, 3584 for math). The model thinks before it answers; truncated mid-thought it produces no final answer at all.
• The bundle declares the thought channel (<think> … </think>), so runtimes that expose ThinkingConfig / a thinking budget can cap or read the reasoning separately; the streamed answer contains only the final response. enable_thinking: false renders the vendor's no-think form (<|Bot|></think>).
• The bundle carries the vendor prompt format as a Jinja template: a default system block (<|System|>\nyou are a helpful assistant., a user system prompt appended after it), every message wrapped in <｜start▁of▁sentence｜> … <｜end▁of▁sentence｜>, generation prompt <|Bot|><think>; stop token <｜end▁of▁sentence｜>. Tool-call formatting is not carried. KV budget 4096 tokens (the original's 1M context does not apply on-device), eleven prefill signatures (1, 2, 4, ..., 1024).
• The vendor's recommended sampling is temperature 1.0 / top-p 0.95; the numbers on this card are greedy.

Performance

Apple M4 Max (litert-lm benchmark 0.17.0, -p 256 -d 256 --runs 3 --cache no, quiet machine, serialized, ≥300 s rest before each GPU reading; every backend first proven to generate on the file):

File	Backend	Prefill (256)	Decode	TTFT	Init
int8	GPU (Metal)	2308 tok/s	95.8 tok/s	0.12 s	3.6 s
int8	CPU	820 tok/s	41.0 tok/s	0.34 s	18.1 s
int4	GPU (Metal)	2492 tok/s	103.6 tok/s	0.11 s	4.1 s
int4	CPU	271 tok/s	38.2 tok/s	0.97 s	5.4 s

Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno; litert_lm_advanced_main from the litert-lm v0.16.0 release kit, 205-token prompt with --benchmark, 2 runs per cell, ranges shown):

File	Backend	Prefill (205)	Decode	TTFT	Init	Peak RSS
int8	GPU (OpenCL)	303–637 tok/s	13.9–15.8 tok/s	0.40–0.78 s	4.5–6.8 s	1098 MB
int8	CPU	165–301 tok/s	12.0–12.3 tok/s	0.79–1.37 s	0.3 s	2606 MB
int4	GPU (OpenCL)	307–459 tok/s	17.4–17.8 tok/s	0.52–0.75 s	10.2–24.4 s	1325 MB
int4	CPU	83–186 tok/s	13.6–13.9 tok/s	1.22–2.65 s	2.9–9.9 s	1961 MB

iPhone 17 Pro (G41DeviceTest harness, composite 8-question prompt, --max-tokens 3072, on-device byte count verified against the source file; score = questions answered correctly inside one reply):

File	Metal GPU	CPU	Init GPU	Init CPU
int8	7/8	7/8	13.0 s	0.25 s
int4	8/8	8/8	5.96 s	2.74 s

The int8 file's one miss on each backend is the rhyme line at the end of the composite 8-question prompt (it answers "purple"); asked on its own the same question is answered "blue" on every backend

Conversion notes

Converted with litert-torch 0.9.3 / transformers 5.14.1 / ai-edge-quantizer, released wheels (reproduction script: hf-to-litertlm).

• The vendor modeling code is patched for export, not re-implemented. modeling_spark.py computes attention through its own eager function and ignores config._attn_implementation; the export copy dispatches through the registered attention interface (so litert-torch's transposed KV cache is used), threads the per-call kwargs, declares the attention-backend capability flags, and applies the per-head sigmoid output gate in the interface's layout. In eager mode the patched file is bit-identical to the vendor file (max |Δlogit| 0.0 on 24 random tokens); two further edits make the vendor file load under transformers 5 at all (_tied_weights_keys mapping form, mask-utility kwargs).
• The think opener is pre-filled by the bundle's template and the thought channel is declared in the metadata. Without the channel the runtime streams raw reasoning into the answer and silently ignores any thinking budget.
• No start token in the metadata. The tokenizer declares <｜start▁of▁sentence｜> as BOS but never prepends it (add_bos_token: false); the template carries its own. The exporter's unconditional start_token write was suppressed (measured harmless in bf16 on the 8-question gate, but it is not the vendor prompt).
• int4 externalizes the embedding table. The vocab is tied; asking int4 for the lm_head and int8 for the embedding makes the quantizer copy the 131,072-row table once per signature (measured 4.6 vs 1.7 bytes/parameter on a tiny checkpoint). The embedder lives in its own section instead. The int8 file needs no split.
• Tokenizer embedded as the upstream tokenizer.json (HF tokenizer section), not a SentencePiece conversion.
• Quantization at export time: int8 = dynamic per-channel on linears + embedding; int4 = int4 blockwise-32 + OCTAV on linears, int8 embedding (externalized). No post-processing.

License and changes

Distributed under Apache-2.0 (inherited from the base model; the upstream LICENSE file is included). Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged into the .litertlm bundle (thinking pre-fill and thought-channel metadata as described); tool-calling template branches not carried. No training or fine-tuning. This repository is a community conversion and is not affiliated with iFLYTEK or the SparkLLM Team.