Text Generation
Transformers
Safetensors
GGUF
English
qwen3_5
image-text-to-text
decision-model
typed-decisions
calibration
calibrated-probabilities
classification
tool-selection
agent-routing
decision-index
jevbench
jev-compatible
systemone
wald
wald-q4b
qwen3.5
4b
vllm
reasoning
llama.cpp
conversational
Eval Results (legacy)
wald-serve 0.2.2: native video input (videos / video_url; video_route native32 default, frames16); decode once per request; card, API, runbook, evaluation/v2.1/video.json; MANIFEST updated. Weights unchanged.
d17e09a verified | license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-4B | |
| base_model_relation: finetune | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| language: | |
| - en | |
| tags: | |
| - decision-model | |
| - typed-decisions | |
| - calibration | |
| - calibrated-probabilities | |
| - classification | |
| - tool-selection | |
| - agent-routing | |
| - decision-index | |
| - jevbench | |
| - jev-compatible | |
| - systemone | |
| - wald | |
| - wald-q4b | |
| - qwen3.5 | |
| - 4b | |
| - vllm | |
| - reasoning | |
| - gguf | |
| - llama.cpp | |
| model-index: | |
| - name: Wald-Q4B v2.1 (056A0-c31, revision v2.1) | |
| results: | |
| - task: | |
| type: text-classification | |
| name: Structured decisions (option probabilities) | |
| dataset: | |
| name: Decision Index 0.3, complete public suite (140,620 requests) | |
| type: decision-index | |
| revision: 62d2f51de34a2de64906345b6bc3e98e27ff55c7 | |
| metrics: | |
| - type: balanced_skill_index | |
| name: Public index, Auto 0.7 (author-run with the official kit; not the organizer's Full score) | |
| value: 60.05 | |
| verified: false | |
| # Wald-Q4B v2.1 | |
| An open 4B decision model: send a state and typed options, get a calibrated probability for every option. Served through a Jev-compatible `POST /v1/systemone` API on your own GPU. | |
| ## Why Wald-4B v2.1 | |
| - **Probabilities, not text:** a calibrated probability for every option, nothing to parse. | |
| - **Decision Index 0.3: 60.05** on the complete public suite (our run; v2.0: 54.18). | |
| - **Thinks only when unsure:** Auto 0.7 thinks natively on about half of the requests. | |
| - **Fast:** median **158 ms** (serial, one request in flight, warm, processing time, one RTX PRO 6000, 200 DI 0.3 requests). | |
| - **Web actions:** Mind2Web element selection **92.3 %** (test splits, 6 candidates; v2.0: 84.6 %). | |
| - **Reads images and video** zero-shot through Qwen3.5-4B's unchanged vision tower. Video (wald-serve 0.2.2): **87.4 %** on VisualDecisionBench video (7,143 questions, our run), 2.5 s for a 15-second clip with 24 questions (serial, one RTX 5090). | |
| - **Small and open:** 4B, Apache-2.0. | |
| ## Scores | |
| | Benchmark | **Wald-Q4B v2.1** | Jev 1.13 (hosted) | | |
| |---|---:|---:| | |
| | Decision Index 0.3, complete public suite | **60.05** (our full run; raw 70.12, breadth 58.63) | 57.96 (board public column) | | |
| | JevBench public set (231) | **210\*** | 200/231 | | |
| | JevBench-XL TEST (8,177) | **67.75 %** | 67.85 % | | |
| | Mind2Web element selection (test task / website / domain) | **94.9 / 91.5 / 91.7 %** | 88.3 / 86.9 / 86.6 % | | |
| | Image decisions, 11 public Decision Index Vision benchmarks (our conversions) | **67.23** | no image input | | |
| | JevAdvBench flip rate, lower is better | **7.1 %** | 6.1 % | | |
| \* Measured on Wald-4B v2.0. | |
| Self-run, not leaderboard results; how each number was produced: [evaluation/v2.1/](https://huggingface.co/org2ai/Wald-4B/tree/main/evaluation/v2.1). DI 0.3 run files: [org2ai/Wald-Q4B-decision-index-results](https://huggingface.co/datasets/org2ai/Wald-Q4B-decision-index-results/tree/main/runs/wald-q4b-v21-056A0-auto07). | |
| ## Quick start | |
| ```sh | |
| hf download org2ai/Wald-4B --revision v2.1 --local-dir ./Wald-Q4B-v2.1 && cd Wald-Q4B-v2.1 | |
| ./run.sh "$PWD" # POST /v1/systemone on :8000 (one NVIDIA GPU, uv) | |
| curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ | |
| "state": "The customer wants to return a damaged kettle.", | |
| "questions": {"route": {"type": "choice", "instructions": "Choose the support queue.", | |
| "criteria": {"returns": "Returns and refunds", "delivery": "Delivery tracking", "other": "Other"}}}}' | |
| curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ | |
| "state": "Photo from the returns desk.", "images": ["data:image/jpeg;base64,..."], | |
| "questions": {"damaged": {"type": "noul", "instructions": "Is the item visibly damaged?"}}}' | |
| curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ | |
| "state": "Aisle camera, last 15 seconds.", "videos": ["data:video/mp4;base64,..."], | |
| "questions": {"conceal": {"type": "noul", "instructions": "Does the shopper put an item into a bag or pocket?"}}}' | |
| ``` | |
| Video needs the server from `main` (wald-serve 0.2.2; the weights are the same as tag `v2.1`): one video per request, up to 32 MB and 300 s. By default the server sends 32 frames to vLLM as one native video (`"video_route": "native32"`). `"video_route": "frames16"` sends 16 labelled frames as images instead, and is used automatically when the request also has images. Details: [docs/api.md](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/api.md#video), numbers: [evaluation/v2.1/video.json](https://huggingface.co/org2ai/Wald-4B/blob/main/evaluation/v2.1/video.json). | |
| API fields: [docs/api.md](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/api.md). Server options, Docker and evaluation settings: [RUNBOOK.md](https://huggingface.co/org2ai/Wald-4B/blob/main/RUNBOOK.md). | |
| ## How it works | |
| - One forward pass reads a probability for every option from the model's option-letter logits (Qwen chat template), calibrated with a frozen temperature table. | |
| - Auto 0.7 (default): if the top probability is below 0.7, the model thinks once natively (at most 512 tokens) and reads the options again. | |
| - More than 26 options: a one-pass knockout over all options. | |
| - Images go through Qwen3.5-4B's vision tower, unchanged (zero-shot). The server decodes a video once per request; by default 32 frames go to the tower as one Qwen3.5 video, with timestamps. | |
| ## Details | |
| - **Training.** One full-parameter pass of decision training from Qwen3.5-4B (chat), with a light KL anchor to the base model; the released weights are 0.85 × the trained model + 0.15 × the base (checkpoint `056A0-c31`). Thoughts are the base model's own; no thought text was trained. | |
| - **Data.** About 23% of training tokens come from the train splits of public benchmarks that the Decision Index also draws on, reformatted as decisions. The rest is broad decision data: general decision tasks — rules and policies, classification, entailment and fact checking, tables and documents, judging (about 35%); hard decisions written and/or labelled by large frontier models, including robustness cases with perturbed states (about 15%); task-specific decision sets (about 11%); web and GUI actions (about 8%); and math and step-by-step reasoning (about 8%). Public datasets are used as train splits only. Contamination scan against the full Decision Index 0.3 public suite: 0 strict hits. | |
| - **Limits.** Scores are self-run; confidence is not correctness, so set thresholds on your own data. | |
| Answers after a thought are sampled and can vary slightly; one pass is deterministic. Thoughts are longer than v2.0's. | |
| - **Licence.** Apache-2.0 for weights and code ([NOTICE](https://huggingface.co/org2ai/Wald-4B/blob/main/NOTICE)). Earlier versions: tags [`v2.0`](https://huggingface.co/org2ai/Wald-4B/tree/v2.0), [`v1.2`](https://huggingface.co/org2ai/Wald-4B/tree/v1.2), [`v1.1`](https://huggingface.co/org2ai/Wald-4B/tree/v1.1), [`v1.0`](https://huggingface.co/org2ai/Wald-4B/tree/v1.0). GGUF of v2.1: [org2ai/Wald-4B-GGUF](https://huggingface.co/org2ai/Wald-4B-GGUF). The larger model in the same family is [org2ai/Wald-26B-A4B](https://huggingface.co/org2ai/Wald-26B-A4B). Cite: [CITATION.cff](https://huggingface.co/org2ai/Wald-4B/blob/main/CITATION.cff). | |