--- license: apache-2.0 base_model: Qwen/Qwen3.5-4B base_model_relation: finetune library_name: transformers pipeline_tag: text-generation language: - en tags: - decision-model - typed-decisions - calibration - calibrated-probabilities - classification - tool-selection - agent-routing - decision-index - jevbench - jev-compatible - systemone - wald - wald-q4b - qwen3.5 - 4b - vllm - reasoning - gguf - llama.cpp model-index: - name: Wald-Q4B v2.1 (056A0-c31, revision v2.1) results: - task: type: text-classification name: Structured decisions (option probabilities) dataset: name: Decision Index 0.3, complete public suite (140,620 requests) type: decision-index revision: 62d2f51de34a2de64906345b6bc3e98e27ff55c7 metrics: - type: balanced_skill_index name: Public index, Auto 0.7 (author-run with the official kit; not the organizer's Full score) value: 60.05 verified: false --- # Wald-Q4B v2.1 An open 4B decision model: send a state and typed options, get a calibrated probability for every option. Served through a Jev-compatible `POST /v1/systemone` API on your own GPU. ## Why Wald-4B v2.1 - **Probabilities, not text:** a calibrated probability for every option, nothing to parse. - **Decision Index 0.3: 60.05** on the complete public suite (our run; v2.0: 54.18). - **Thinks only when unsure:** Auto 0.7 thinks natively on about half of the requests. - **Fast:** median **158 ms** (serial, one request in flight, warm, processing time, one RTX PRO 6000, 200 DI 0.3 requests). - **Web actions:** Mind2Web element selection **92.3 %** (test splits, 6 candidates; v2.0: 84.6 %). - **Reads images and video** zero-shot through Qwen3.5-4B's unchanged vision tower. Video (wald-serve 0.2.2): **87.4 %** on VisualDecisionBench video (7,143 questions, our run), 2.5 s for a 15-second clip with 24 questions (serial, one RTX 5090). - **Small and open:** 4B, Apache-2.0. ## Scores | Benchmark | **Wald-Q4B v2.1** | Jev 1.13 (hosted) | |---|---:|---:| | Decision Index 0.3, complete public suite | **60.05** (our full run; raw 70.12, breadth 58.63) | 57.96 (board public column) | | JevBench public set (231) | **210\*** | 200/231 | | JevBench-XL TEST (8,177) | **67.75 %** | 67.85 % | | Mind2Web element selection (test task / website / domain) | **94.9 / 91.5 / 91.7 %** | 88.3 / 86.9 / 86.6 % | | Image decisions, 11 public Decision Index Vision benchmarks (our conversions) | **67.23** | no image input | | JevAdvBench flip rate, lower is better | **7.1 %** | 6.1 % | \* Measured on Wald-4B v2.0. Self-run, not leaderboard results; how each number was produced: [evaluation/v2.1/](https://huggingface.co/org2ai/Wald-4B/tree/main/evaluation/v2.1). DI 0.3 run files: [org2ai/Wald-Q4B-decision-index-results](https://huggingface.co/datasets/org2ai/Wald-Q4B-decision-index-results/tree/main/runs/wald-q4b-v21-056A0-auto07). ## Quick start ```sh hf download org2ai/Wald-4B --revision v2.1 --local-dir ./Wald-Q4B-v2.1 && cd Wald-Q4B-v2.1 ./run.sh "$PWD" # POST /v1/systemone on :8000 (one NVIDIA GPU, uv) curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "The customer wants to return a damaged kettle.", "questions": {"route": {"type": "choice", "instructions": "Choose the support queue.", "criteria": {"returns": "Returns and refunds", "delivery": "Delivery tracking", "other": "Other"}}}}' curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Photo from the returns desk.", "images": ["data:image/jpeg;base64,..."], "questions": {"damaged": {"type": "noul", "instructions": "Is the item visibly damaged?"}}}' curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Aisle camera, last 15 seconds.", "videos": ["data:video/mp4;base64,..."], "questions": {"conceal": {"type": "noul", "instructions": "Does the shopper put an item into a bag or pocket?"}}}' ``` Video needs the server from `main` (wald-serve 0.2.2; the weights are the same as tag `v2.1`): one video per request, up to 32 MB and 300 s. By default the server sends 32 frames to vLLM as one native video (`"video_route": "native32"`). `"video_route": "frames16"` sends 16 labelled frames as images instead, and is used automatically when the request also has images. Details: [docs/api.md](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/api.md#video), numbers: [evaluation/v2.1/video.json](https://huggingface.co/org2ai/Wald-4B/blob/main/evaluation/v2.1/video.json). API fields: [docs/api.md](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/api.md). Server options, Docker and evaluation settings: [RUNBOOK.md](https://huggingface.co/org2ai/Wald-4B/blob/main/RUNBOOK.md). ## How it works - One forward pass reads a probability for every option from the model's option-letter logits (Qwen chat template), calibrated with a frozen temperature table. - Auto 0.7 (default): if the top probability is below 0.7, the model thinks once natively (at most 512 tokens) and reads the options again. - More than 26 options: a one-pass knockout over all options. - Images go through Qwen3.5-4B's vision tower, unchanged (zero-shot). The server decodes a video once per request; by default 32 frames go to the tower as one Qwen3.5 video, with timestamps. ## Details - **Training.** One full-parameter pass of decision training from Qwen3.5-4B (chat), with a light KL anchor to the base model; the released weights are 0.85 × the trained model + 0.15 × the base (checkpoint `056A0-c31`). Thoughts are the base model's own; no thought text was trained. - **Data.** About 23% of training tokens come from the train splits of public benchmarks that the Decision Index also draws on, reformatted as decisions. The rest is broad decision data: general decision tasks — rules and policies, classification, entailment and fact checking, tables and documents, judging (about 35%); hard decisions written and/or labelled by large frontier models, including robustness cases with perturbed states (about 15%); task-specific decision sets (about 11%); web and GUI actions (about 8%); and math and step-by-step reasoning (about 8%). Public datasets are used as train splits only. Contamination scan against the full Decision Index 0.3 public suite: 0 strict hits. - **Limits.** Scores are self-run; confidence is not correctness, so set thresholds on your own data. Answers after a thought are sampled and can vary slightly; one pass is deterministic. Thoughts are longer than v2.0's. - **Licence.** Apache-2.0 for weights and code ([NOTICE](https://huggingface.co/org2ai/Wald-4B/blob/main/NOTICE)). Earlier versions: tags [`v2.0`](https://huggingface.co/org2ai/Wald-4B/tree/v2.0), [`v1.2`](https://huggingface.co/org2ai/Wald-4B/tree/v1.2), [`v1.1`](https://huggingface.co/org2ai/Wald-4B/tree/v1.1), [`v1.0`](https://huggingface.co/org2ai/Wald-4B/tree/v1.0). GGUF of v2.1: [org2ai/Wald-4B-GGUF](https://huggingface.co/org2ai/Wald-4B-GGUF). The larger model in the same family is [org2ai/Wald-26B-A4B](https://huggingface.co/org2ai/Wald-26B-A4B). Cite: [CITATION.cff](https://huggingface.co/org2ai/Wald-4B/blob/main/CITATION.cff).