GLM-5.3-Flash — W4A16 (INT4) + BF16 MTP

INT4 weight-only quantization of zai-org/GLM-5.3-Flash with the BF16 MTP draft head preserved for speculative decoding. 177.7 GiB (−70%) — serves on 4× H100/H200, 4× RTX PRO 6000, or 2× DGX Spark at full 1M context.

🏆 Headline — where W4A16 wins (all measured, same-harness comparisons)

  • −70% weights: 177.7 GiB vs ~599 GiB BF16 → 4 datacenter GPUs instead of 8, or a 2× DGX Spark desktop pair at full 1M context.
  • vs NVFP4 (LibertAI, strongest community quant) — 2× DGX Spark: W4A16 serves 1,048,576 tokens there (KV pool 1.30×); the NVFP4 checkpoint OOM'd 9/9 boots at any context on identical nodes (dmesg-verified). Throughput @262K: +53% seq1 / +101% seq6.
  • vs NVFP4 — H100 TP=4, matched MTP: wins c1 +4.8% · c8 +13.6% · saturated c32 +14.0% (vs NVFP4's best c32 config); mtp-off fixed-drafter c32 +22.4% — the batch edge is the marlin INT4 MoE GEMM, not the drafter.
  • vs EXL3 (MiaAI-Lab 2×-Spark recipe) — independent matched-protocol head-to-head, FINAL: W4A16 leads single-stream +10.4% (33.0 vs 29.9 tok/s) and long-prefill +31–39% (~1.2K vs 864–914 tok/s @300K-class); KV capacity 1.30× vs 1.20×. The vendor README overstated decode ~2.2× and prefill ~1.2× vs independent measurement.
  • Quality: AIME 2026 85.0% (102/120, n=120 final, max thinking)beats the matched EXL3 arm 96/120 (80.0%) by +6 correct (+5.0 pt), FINAL 2026-09-06; AIME 2025 parity with NVFP4 on H100 (0.8833 vs 0.9000, 0.42σ); GSM8K 0.97; GPQA-Diamond within noise.
  • SM120 (Blackwell): throughput parity with NVFP4 (±0.7%) with MTP paying at every concurrency — +69.9% c1 / +37.6% c8 / +47.8% c32.
  • Long context: single-stream decode is KV-length-insensitive to ≥486K (up to 357 tok/s, bf16 KV); long-prefill ceiling ~6.1–8.5K tok/s aggregate across the full 3-KV × c1–c32 matrix.
  • Research-stack bug found and fixed: a 21-leg investigation root-caused a hang-forever admission wedge in the DFlash2 spec-decode stack and validated a fix to 256,388-token prompts (24/24 quality spot-checks pass).

Honest ledger: EXL3 leads mid-concurrency (c2 +67% / c4 +89%; ≈+81% cap-corrected c6); both sides saturate at c32 (EXL3 881–898 agg tok/s). SM120 vs NVFP4 is parity. The matched AIME-2026 quality arm vs EXL3 is FINAL 2026-09-06: W4A16 102/120 (85.0%) vs EXL3 96/120 (80.0%) — W4A16 wins. Results as of 2026-09-06; in-flight arms are labeled.

Why this quant exists

  • BF16 GLM-5.3-Flash needs eight 80 GB GPUs per replica; this artifact cuts weights to 177.7 GiB — four GPUs, or a dual DGX Spark desktop at 1M context.
  • Only the 36,288 routed-expert GEMMs are INT4 (symmetric, group-128, GPTQ); attention, router, shared experts, embeddings, the BF16 vision tower, and the BF16 MTP draft head (+40–50% single-stream) stay unquantized; canonical FP32 tensors preserved verbatim.
  • Not a new model — all capability comes from the base model.

Hardware

Config GPUs Context Notes
H100/H200 (SM90), TP=4 4× 80 GB 262K, bf16 KV bench-validated recipe
H200 (SM90), TP=2 2× 141 GB ~64–128K, fp8 KV fit-estimated (not rig-measured)
RTX PRO 6000 (SM120), TP=4 4× 96 GB 262K, fp8 KV bench-validated
DGX Spark GB10 (SM121), TP=2 2× 128 GB UMA 1M, fp8 KV DFlash2 stack — recipe below
B300 (SM103) quantization node (transformers smoke-tested)

Benchmarks vs competitors

Identical flags/prompts/hardware per row. Throughput = output tok/s, thinking ON.

vs EXL3 (MiaAI-Lab) — 2× DGX Spark, matched protocol, FINAL (2026-09-03)

  • EXL3 was brought to full serving at 1M context on the same hardware (KV pool 1,200,348 = 1.20×; W4A16 1,360,420 = 1.30×).
  • Like-for-like decode (TTFT excluded, 8K prompts, 256-token completions): W4A16 +10.4% c1 (33.0 vs 29.9 agg tok/s).
  • EXL3 leads mid-concurrency: c2 +67.4% (59.6 vs 35.6), c4 +89.1% (112.7 vs 59.6), c6 ≈+80.9% after correcting its max-num-seqs=4 scheduler cap; it sustains ~29 tok/s per stream at every concurrency (W4A16 steps down 33.0→11.2 seq1→6 on the 1M serve; the 262K serve holds ~20 at seq4+).
  • Saturated c32 (clean refire): EXL3 881–898 agg tok/s (32/32 streams, per-stream 27.4–28.1).
  • W4A16 +31–39% long-prefill (~1.2K tok/s vs 864–914 @300K-class prompts).
  • Vendor README overstated measured decode ~2.2× (claimed 62.9/51.7/37.1 per-stream c1/2/4 vs measured 26.6/21.2/17.0) and prefill ~1.2×.
  • Two mid-bench hardware deaths on the competitor node were quarantined and never banked; the c32 set was re-run clean.
  • Matched AIME-2026 quality arm: FINAL 2026-09-06 — W4A16 102/120 (85.0%) beats EXL3 96/120 (80.0%), +6 correct (+5.0 pt). EXL3 epochs 25/25/24/22, zero harness errors; 23/120 EXL3 rows hit the 131,072-token cap (W4A16: 16/18 misses were budget-exhaustion) — scored as written under the identical temp-0.0 / 131,072-token / n=120 config as our banked 85.0% arm.

vs NVFP4 — H100/H200 (SM90), TP=4, MTP both sides, isl/osl 8192/1024

concurrency W4A16 MTP N=2 (this) NVFP4 MTP N=2 NVFP4 MTP N=5 (its best c32 config)
1 183.89 180.96 175.44
8 249.94 251.14 220.05
32 1161.13 770.66 1018.13
MTP acceptance 52–55% ~52% ~30%
  • Headline: +4.8% c1 / +13.6% c8 / +14.0% saturated c32 vs NVFP4's banked N=5 config.
  • † c=32 with --gpu-memory-utilization 0.92 (KV-pool capacity): 1161.13 tok/s (independent tuned run: 1145.05). The c=32 lever is KV-pool size, not a kernel gap.
  • Fixed-drafter 2×2 re-confirm (both arms on the corrected #54374-port overlay): matched c1/c8 within noise (180.22/254.39 vs 182.13/252.89); saturated c32 win holds (+18.1%, 804.02 vs 680.99).
  • MTP-off on both: W4A16 leads +2.7% / +10.5% / +22.4% at c1/c8/c32 — the c32 advantage is the marlin INT4 MoE GEMM at batch, not an MTP artifact.
  • MTP adds +41.1%/+14.3%/−4.0% (W4A16) and +46.4%/+25.6%/−0.4% (NVFP4) at c1/c8/c32 on SM90.

vs NVFP4 — RTX PRO 6000 (SM120, Blackwell), TP=4, matched MTP N=2

concurrency W4A16 (this) NVFP4 Δ
1 109.90 109.34 +0.5%
8 318.54 319.46 −0.3%
32 534.37 530.75 +0.7%
  • Parity within noise (±0.7%); verbatim-methodology re-grid: c1 106.97 (−2.7%, 8-prompt high-variance class), c8 320.51 (+0.6%), c32 539.63 (+1.0%), MTP acceptance 50–53% — no stack drift.
  • MTP pays at every concurrency on SM120: +69.9% c1 (106.97 vs 62.96), +37.6% c8 (320.51 vs 232.89), +47.8% c32 (539.63 vs 365.02) — a sign flip vs SM90 (drafter costs ~4% at saturated c32 there). Keep MTP on at every concurrency on SM120.
  • KV-dtype at batch: fp8 vs bf16 costs nothing at c1/c8 (±0.05%) and ≈4.7% at c32 (365.02 vs 382.07) — lever ranking MTP ≫ KV dtype. (auto is the working bf16-KV path; bf16/bfloat16 variants fail on this stack.)

Long-context decode at batch (SM120, TP=4, 524K ctx, MTP N=2 — decode-isolated, TTFT excluded)

Per-stream decode tok/s (full-contention cohort bands — serialized prefills split stages into early/late cohorts, so bands, not blended medians):

KV length KV dtype c1 c4 c8 c32
8K (aggregate grid) fp8 106.97 40.06 16.86
138.6K fp8 143.94 2.92–7.53 2.49–3.74 2.24–2.72
138.6K bf16 (auto) 152.19 3.23–4.21 2.53–2.64 2.45–2.63
277K bf16 (auto) 177.15 2.23–6.15 2.28–4.32 2.17–2.79
486K bf16 (auto) 357.34* 2.27–3.32 2.31–2.68 2.28–4.27†
486K fp8 (degenerate‡) 2.31–3.26 2.37–3.62 2.03–3.93†

* n=1 short completion, noisy. † 21/32 rows — 11 bench-client TimeoutErrors disclosed in-repo. ‡ the fp8-KV c1 row recorded an absurd >480K tok/s (bench artifact) and is never cited. Additional bands: 277K c2 5.03, c6 2.29–3.38; 486K c6 2.34–3.68 (bf16) / 2.34–3.60 (fp8); 138.6K fp8 c2 9.58.

  • Single-stream decode is KV-length-insensitive to ≥486K (143.94 → 177.15 → 357.34 tok/s at 138.6K/277K/486K bf16 — rises, if anything).
  • At-batch collapse saturates with KV length: 13× per-stream drop at c8 vs the 8K grid, then flat ~2.2–3.7 tok/s/stream at c4–c32 from 138.6K through 486K — mechanism is target-model forward compute at batch × long-KV (4.4M tokens of live KV at c32), not the drafter.
  • KV dtype is a c1-only ~5% lever (bf16 +5.7% @138.6K); at-batch bands overlap fp8/bf16 — the long-KV-amplifies-fp8-cost hypothesis is refuted.
  • MTP acceptance never collapses: c1 ≈80–86% → c8 73.9% (n=1,286) → c32 71.6% (n=4,714) at 138.6K; 60.0% single-stream at 486K (small sample, disclosed). The saturated c32 stage decoded 2.53 tok/s/stream while the drafter accepted 71.6% — conclusively not acceptance-driven.
  • At-batch lever ranking FINAL: MTP drafter ≫ everything measured; KV dtype ≈ c1-only ~5%; drafter acceptance is not the bottleneck.

Long-prefill at batch (SM120) — full 3-KV × c1–c32 matrix COMPLETE

  • Single-stream prefill: ~6,100–7,100 tok/s for 277K–486K prompts (TTFT first request 39.2–79.3 s).
  • At batch, long prefills are essentially serialized (stage wall ≈ N × single-stream time).
  • Aggregate prefill ceiling ~6.1–8.5K tok/s — KV-length-insensitive AND batch-insensitive:
  • 138.6K: 6,735 (c1) → 7,134 (c8) → 7,897 (c16) → 8,486 (c32) tok/s.
  • 277K: 5,211 (c1) → 7,177 (c8) → 7,784 (c16) → 7,357 (c32) tok/s.
  • 486K: 6,136 (c1) → 6,892 (c8) → 7,483 (c16) → 6,988.6 (c32) tok/s.
  • Serialization ratios (wall / N×c1): 138.6K 0.94/0.85/0.79 · 277K 0.73/0.67/0.71 · 486K 0.89/0.82/0.88 (c8/c16/c32) — deeper-KV rows behave closer to strict serialization.
  • mnb lever fully mapped: 8192 ✓ (validated production config) → 16384 ✓ (validated fallback, ceiling within ~1% at both 277K and 486K) → 32768 ✗ (CUDA OOM at 486K — 3.98 GiB needed vs 1.70 GiB free at mem-util 0.95). Ceiling is chunk-size-insensitive → attention/forward compute-bound, not scheduling-bound.
  • Operational guidance: admission pacing / prefill scheduling is the only lever that moves long-context batch UX (mnb tuning and MTP do not address prefill serialization).

2× DGX Spark GB10 (SM121), TP=2, DFlash2 block-diffusion spec-decode

8K prompts, 256-token completions, aggregate tok/s:

config seq1 seq2 seq4 seq6
W4A16+DFlash2 @262K (this) 33.29 42.01 80.98 121.81
NVFP4-MTP4 reference 21.8 41 66 60.5
Δ +53% +2.4% +23% +101%
W4A16+DFlash2 @1M serve 32.99 35.61 59.61 67.05
  • 256K-token single prompt on the 1M serve: 30.85 tok/s decode (ttft 5.8 s).
  • Structural: the NVFP4 checkpoint (90.67 GiB/node weights vs ~30 for W4A16) OOM'd all 9 boot attempts across 1M/300K/262K on the 121 GiB unified-memory nodes (dmesg-verified) — it cannot serve at any usable context on 2× Spark.

Quality

  • AIME 2026 (30 problems, max thinking, 131,072-token budget, DFlash2 stack, 2× Spark): 102/120 = 85.0% — n=120 FINAL. Epochs 28/30, 25/30, 26/30, 23/30; 16/18 misses are budget-exhaustion truncations, 2/18 genuine wrong answers. The LibertAI NVFP4 external AIME-2025 reference (0.9000) remains +5.0 points ahead; the matched AIME-2026 EXL3 arm is FINAL 2026-09-06: 96/120 (80.0%) — W4A16 wins by +6 correct (+5.0 pt).
  • AIME 2025 n=120, 4×30, max thinking, 131,072 budget — H100 (SM90): parity — W4A16 0.8833 (106/120) vs NVFP4 0.9000 (0.42σ). An earlier 0.7833 read was a serving-stack drafter-corruption bug (upstream vLLM #54374), since fixed.
  • AIME 2025 — RTX PRO 6000 (SM120): raw-arm deficit real (~2σ; 0.8083/0.8000 pooled vs 0.9000), mechanism decomposed and fix VALIDATED. ≈63% of the deficit is a mechanical budget-wall (EMPTY-answer rate 11.7–14.2% vs NVFP4's 3.3%; doubling the budget recovered +5.8 pt); ≈37% is SM120 kernel numerics (near-tie argmax flips vs SM90, logit-quantified).
  • Fix stack VALIDATED (zero GPU cost): whole-trace commit hook + thinking-budget force-commit (98,304-token cap) lifts the score to 0.8833 two-way / 0.8917 hook-credited — within noise of NVFP4 0.9000 (0.42σ / 0.21σ).
  • Production-overlay pilot passed (committed 0.8917, zero modified non-empty responses across 120 requests); production adoption remains gated on the eval sign-off chain (owner decision pending).
  • GSM8K: parity — 0.9704–0.9750 across all three quants on both x86 rigs.
  • GPQA-Diamond n=198 @131k: 0.8586 vs 0.8687 NVFP4 (SM90), 0.8586 vs 0.8737 (SM120) — within noise.
  • Matched AIME-2026 single-epoch arms on x86 (n=30, max thinking): W4A16 24/30 (RTX PRO 6000) and 22/30 (H100) vs NVFP4 23/30 — within noise at n=30.

Serving

The same checkpoint serves on every NVIDIA architecture from Hopper through Blackwell. Each recipe is the exact config the benchmarks ran.

SM90 — H100 / H200 (recommended: TP=4)

Image vllm/vllm-openai:glm53-flash-x86_64-cu130 (CUDA 13.0). Gotcha: always pass --max-num-seqs ≤512 — the default 1024 exceeds this hybrid linear-attention model's 512 Mamba/KDA-state cache blocks.

docker run --gpus '"device=0,1,2,3"' --ipc=host --network=host --rm \
  -v /models:/models vllm/vllm-openai:glm53-flash-x86_64-cu130 \
  vllm serve /models/glm53-flash-w4a16-mtp \
    --served-model-name glm53-w4 \
    --tensor-parallel-size 4 --enable-expert-parallel \
    --max-model-len 262144 --max-num-seqs 512 \
    --block-size 64 --gpu-memory-utilization 0.92 --no-enable-prefix-caching \
    --speculative-config '{"method":"mtp","num_speculative_tokens":2}' \
    --reasoning-parser glm45 --tool-call-parser glm47 --enable-auto-tool-choice \
    --trust-remote-code --port 8000
  • MTP num_speculative_tokens: 2 is the sweet spot on Hopper (52–55% acceptance, best c1/c8); N=5 helps only at c=32. TP=8 works too.

SM120 — RTX PRO 6000 (Blackwell)

Image cstechdev/vllm:glm53-flash-nope-sm120-cu130-20260826-r1 (SM120-patched build).

docker run --gpus '"device=4,5,6,7"' --ipc=host --network=host --rm \
  -v /models:/models cstechdev/vllm:glm53-flash-nope-sm120-cu130-20260826-r1 \
  vllm serve /models/glm53-flash-w4a16-mtp \
    --served-model-name glm53-w4 \
    --tensor-parallel-size 4 --enable-expert-parallel \
    --max-model-len 262144 --max-num-seqs 64 --max-num-batched-tokens 8192 \
    --kv-cache-dtype fp8 --gpu-memory-utilization 0.92 --enable-prefix-caching \
    --speculative-config '{"method":"mtp","num_speculative_tokens":2}' \
    --reasoning-parser glm45 --tool-call-parser glm47 --enable-auto-tool-choice \
    --trust-remote-code --port 8001
  • KV fp8 is required at 262K on 96 GB cards; --max-num-seqs 64 is bench-validated (48/36 fallback ladder); MTP N=2 matches NVFP4 to ±0.7%.

SM121 — 2× DGX Spark GB10 (desktop, 1M context)

Full 1,048,576-token context on two Sparks over RoCE (TP=2) with the DFlash2 block-diffusion drafter:

  • Image radixark/vllm-glm53-flash:sm121-v11-dflash2; drafter GLM-5.3-Flash-DFlash2 (block 8, selector_rank 256, top_k 16, layers [5,14,24,33,42]).
  • MAX_MODEL_LEN=1048576 KV_CACHE_MEM=9663676416 → KV fp8_e4m3 pool 1,360,420 tokens (1.30× a full 1M request); staging @262K = MAX_MODEL_LEN=262144 KV_CACHE_MEM=3221225472.
  • --max-num-seqs 6 --block-size 2304 --enforce-eager; 7 speculative tokens (any other count boot-wedges).
  • Worker rank first, 25 s, then head rank; cold boot ≈6–8 min.
  • NCCL on the RoCE switch path (97.98 Gbps line-rate; direct-attach measured worse).
  • VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=3600 (cold JIT otherwise kills the boot).
  • Keep single prompts ≤ ~310K tokens (larger wedged a host twice).

Docker images

Upstream note (2026-09-03): GLM-5.3-Flash support landed in vLLM main (vllm-project/vllm#53906, merged 16:40 UTC). The pinned images below remain the validated serving stacks for this quant; upstream-main validation is a separate, queued leg.

Architecture Image
SM90 (H100/H200) vllm/vllm-openai:glm53-flash-x86_64-cu130
SM120 (RTX PRO 6000) cstechdev/vllm:glm53-flash-nope-sm120-cu130-20260826-r1
SM121 (DGX Spark) radixark/vllm-glm53-flash:sm121-v11-dflash2

Quick start

# 1. Download the weights (~178 GiB)
huggingface-cli download canada-quant/glm-5.3-w4a16-mtp --local-dir /models/glm53-flash-w4a16-mtp

# 2. Serve (SM90 example — see recipes above)
docker run --gpus '"device=0,1,2,3"' --ipc=host --network=host --rm \
  -v /models:/models vllm/vllm-openai:glm53-flash-x86_64-cu130 \
  vllm serve /models/glm53-flash-w4a16-mtp --served-model-name glm53-w4 \
    --tensor-parallel-size 4 --enable-expert-parallel --max-num-seqs 512 \
    --max-model-len 262144 --speculative-config '{"method":"mtp","num_speculative_tokens":2}' \
    --reasoning-parser glm45 --tool-call-parser glm47 --enable-auto-tool-choice \
    --trust-remote-code

# 3. Call it (OpenAI-compatible)
curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "glm53-w4",
  "messages": [{"role": "user", "content": "Prove there are infinitely many primes."}]
}'

Details

Field Value
Base model zai-org/GLM-5.3-Flash (BF16)
Architecture Glm5NextForConditionalGeneration (glm5_next) — 45 decoder layers (+ MTP layer 45), hidden 4096, 288 routed experts (top-8) + 1 shared, MoE-intermediate 2048, KDA + DSA attention, 24-block vision tower, vocab 154,880
Weight quantization W4A16, INT4, symmetric, group-size 128 (GPTQ, compressed-tensors pack-quantized), routed experts only
Quantized tensors 36,288 (42 MoE layers × 288 experts × 3 GEMMs) — verified exact
Kept in BF16 attention (KDA + DSA incl. indexer), dense prefix (layers 0–2), shared experts, router/gate, mHC tensors, embeddings, lm_head, norms, vision tower (348 keys), MTP layer 45 (889 keys)
Canonical FP32 preserved A_log, dt_bias, e_score_correction_bias, hc_* — verbatim from source
Calibration 256 samples × 4096 tokens, in-distribution chat/code mix (glm53_mix), sequential per-layer GPTQ
Size 177.7 GiB (from ~599 GiB BF16)
License MIT (inherited from the base model)

vLLM serving note: the checkpoint's quantization_config.ignore already excludes the vision tower and re:model\.language_model\.layers\.45\..*, so the MTP head loads BF16 for speculative decoding out of the box.

Validation

Static gates at build time (2026-08-27), all PASS:

  • exactly 36,288 packed tensors; zero compression outside routed experts;
  • vision key-set 348/348 identical to source, zero compressed, all dtypes match;
  • MTP layer present (889 keys), index rewrite purely additive;
  • dtype drift vs source: zero (282 pipeline-drifted tensors restored verbatim);
  • expert scale audit: no zero/collapsed scales.

Quantized on 8× NVIDIA B300, 2026-08-27. B300 smoke tests (transformers ≥5.16): loads as Glm5NextForConditionalGeneration (~110 s); text generation coherent; image captioning accurate.

Vision-quality note: the vision tower is BF16-passthrough but was not covered by the text-only calibration set. Image smoke tests pass; a vision eval leg (MMMU-class + OCR) is queued.

Research-stack known issue: DFlash2 admission wedge — root-caused across 21 legs; FIX VALIDATED (2026-09-04)

Affects the SM90 research stack only (x86_64, DFlash2 drafter n=7). The standard MTP serve path used by all production configs above is unaffected.

  • Symptom: prompts above a block-size-dependent bound are never admitted — Waiting: 1 / Running: 0, 0% KV, retry-forever until timeout (no crash, no OOM).
  • Measured bounds: block 2304 → wedges above ~15.5K tokens; block 1024 → ~17.4K; block 1536 → no wedge at all across the tested 16–24K class.
  • Mechanism (pinned): spec-decode admission requires a full-sequence reservation against the unified KV pool (1,012 blocks); the binding group is the drafter's SlidingWindowManager (block-16) — a 15,523-token request needs exactly 1,011 blocks (passes, zero margin), a ~15,556-token request needs 1,013 (wedges).
  • Root cause (pinned): the drafter SW manager allocates execution-side at full-sequence scale (cdiv(tokens,16)) while its true sliding-window need is only 130 blocks; the built-in skip-free mechanism fires but freed blocks are instantly re-consumed by the next full-sequence allocation.
  • Fix VALIDATED: cap the SW manager's execution allocation at min(cdiv(tokens, bs), cdiv(sliding_window, bs) + 2) (= 130 blocks) — serves 8,528 / 15,787 / 25,344 / 63,734 / 127,952 / 256,388 tokens, all PASS at the shipping block-2304 config (pre-fix wedge at ≥15,537), zero engine errors.
  • Quality spot-suite PASS: 24 deterministic wedge-class prompts, fixed stack vs spec-OFF control — 24/24 checks both arms, zero tokenization mismatch; no detectable quality impact at spot scope.
  • Workarounds without the patch: block 1536 serves the whole tested 16–24K class (its SW cap saturates) and holds the largest KV pool (1.12× of a 262K request vs 0.78× at block 2304).
  • Upstream status: the stock upstream DFlash2 image cannot boot this drafter at all, so the defect is provable only on the research overlay; the closest upstream PR (#50169) would not fix it (no execution-side cap). A fully-evidenced issue draft is finalized in-repo (filing = owner decision); production adoption of the patch = owner decision.
  • Full leg chain: findings/df2w*.md (DF2W → DF2W27).

License

MIT, inherited from the base model. Follow the base model's usage terms.


Built, benchmarked, and documented with the Digby.ai coding harness, developed by CQL.ca.

Downloads last month
781
Safetensors
Model size
50B params
Tensor type
F32
·
I32
·
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for canada-quant/glm-5.3-w4a16-mtp

Quantized
(96)
this model