munod commited on
Commit
7785db1
·
verified ·
1 Parent(s): 86804a7

Upload folder using huggingface_hub

Browse files
README.md CHANGED
@@ -27,8 +27,26 @@ pipeline_tag: text-classification
27
  > [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi)); measured numbers below come
28
  > from `benchmarks/report.md`.
29
  >
30
- > **This revision is B-15: both checkpoints retrained on `template_split: "train"`, and the
31
- > holdout number is honest for the first time.** The data recipe was corrected end to end
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  > (sampler strides decoupled so every language trains every label, evals made
33
  > leave-one-template-out, phrase content ×3, volume 1.7×) and every training set regenerated so
34
  > training and the holdout eval share **0%** of templates. The multilingual lineage carries the
@@ -142,9 +160,9 @@ interleaved touch-up (`training/interleave_continue.py`).
142
  | English (ModernBERT-large + five-domain LoRA r=16 + choice-head bank) | support | **1.0000** | 0.188 | 57.7 |
143
  | English, five-domain split | 5 domains | **0.9983** | 0.191 | 57.7 |
144
  | English, **holdout phrasing** (never in training) | 5 domains | **0.9825** | 0.188 | 58.0 |
145
- | Multilingual (mmBERT-base + LoRA r=64 + fitted bank) — routed runtime | support | **0.9107** | 0.051 | 47.4 |
146
- | Multilingual, five-domain split | 5 domains | **0.9156** | 0.019 | 48.0 |
147
- | Multilingual, **holdout phrasing** (never in training) | 5 domains | **0.8355** | 0.103 | 47.8 |
148
 
149
  The English ECE column reads the **pooled never-trained fit** (below): `choice` is flattened to
150
  T=10.0 to calibrate held-out rows, which shows up honestly in-domain (0.5243 `choice` ECE —
@@ -214,8 +232,11 @@ rows fall through to the English checkpoint by language routing, `.specs/project
214
  0.05 ECE target on that split, and on the **holdout** the B-1 per-language ECE criterion was
215
  **NOT met** at B-15 (7 cells, worst `score/it` 0.1495) — the **B-16** cycle (serve-key
216
  detector + fitted confidence, 2026-10-09) moved it to **11/12 with `score/it` 0.0521
217
- recorded NOT met**, `choice` 0.9248 and every other axis byte-identical (recorded with
218
- numbers in BACKLOG **B-1**/**B-16**) — open under NFR-C06 / BACKLOG B-1.
 
 
 
219
  The CUDA-graph fast path
220
  (`TACHYONE_FAST=1`) gives **2.53×** p50 on English (17.31 → 6.83 ms) and **3.71×** on the
221
  multilingual five-domain path (14.23 → 3.84 ms) — re-measured on the v0.9.0 artifacts
 
27
  > [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi)); measured numbers below come
28
  > from `benchmarks/report.md`.
29
  >
30
+ > **This revision is B-17: the teacher expanded the tone banks, the multilingual arm was
31
+ > retrained, and B-1's per-language ECE acceptance closes.** **352 phrases** authored by a
32
+ > local `qwen3.6:35b` teacher went into every `neutral`/`request` bank (five domains × six
33
+ > languages, inserted **before** the held-out phrase — the holdout evals are byte-identical,
34
+ > sha-pinned); the multilingual checkpoint retrained with the unchanged B-15 recipe and
35
+ > **English was not touched** (byte-identical numbers). On phrasing never seen in any
36
+ > training: **multilingual 0.9264 (routed), English 0.9825** — the in-template → holdout gap
37
+ > collapsed **−0.1105 → −0.0165** — and **all 12 `(primitive, language)` cells now sit at
38
+ > ECE ≤ 0.05** (worst `score/fr` 0.0257; `score/it` **0.0521 → 0.0014**, accuracy
39
+ > 0.8699 → 0.9928) — **B-1 criterion 3 (NFR-C06) CLOSED** on the frozen yardstick. Also
40
+ > improved: `choice` holdout 0.9248 → 0.9808, the `noul` tripwire 0.7807 → 0.9735, every
41
+ > language ≥ 0.9928 (`es` 1.0000), in-template multilingual ECE 0.0228 → 0.0036 (zero gate
42
+ > exceptions everywhere). Disclosures, never smoothed: the expansion targeted a diagnosed
43
+ > mechanism (BACKLOG **B-17** carries the per-domain table); off-domain **MASSIVE
44
+ > 0.074 → 0.058 with `ECE raw` 0.046 → 0.197** — in-domain gains re-sharpened off-domain
45
+ > confidence (the L-007/L-015 trade), still 3.4× its chance; and the English P3 `choice`
46
+ > ECE **0.5243** (pooled fit at T = 10.0) stands exactly as previously disclosed.
47
+ >
48
+ > **The v0.9.0/v0.10.0 revisions were B-15/B-16 (history).** B-15 retrained both checkpoints
49
+ > on `template_split: "train"` after the data recipe was corrected end to end
50
  > (sampler strides decoupled so every language trains every label, evals made
51
  > leave-one-template-out, phrase content ×3, volume 1.7×) and every training set regenerated so
52
  > training and the holdout eval share **0%** of templates. The multilingual lineage carries the
 
160
  | English (ModernBERT-large + five-domain LoRA r=16 + choice-head bank) | support | **1.0000** | 0.188 | 57.7 |
161
  | English, five-domain split | 5 domains | **0.9983** | 0.191 | 57.7 |
162
  | English, **holdout phrasing** (never in training) | 5 domains | **0.9825** | 0.188 | 58.0 |
163
+ | Multilingual (mmBERT-base + LoRA r=64 + fitted bank) — routed runtime | support | **0.9300** | 0.038 | 52.8 |
164
+ | Multilingual, five-domain split | 5 domains | **0.9221** | 0.026 | 53.5 |
165
+ | Multilingual, **holdout phrasing** (never in training) | 5 domains | **0.9264** | 0.032 | 53.3 |
166
 
167
  The English ECE column reads the **pooled never-trained fit** (below): `choice` is flattened to
168
  T=10.0 to calibrate held-out rows, which shows up honestly in-domain (0.5243 `choice` ECE —
 
232
  0.05 ECE target on that split, and on the **holdout** the B-1 per-language ECE criterion was
233
  **NOT met** at B-15 (7 cells, worst `score/it` 0.1495) — the **B-16** cycle (serve-key
234
  detector + fitted confidence, 2026-10-09) moved it to **11/12 with `score/it` 0.0521
235
+ recorded NOT met** — and the **B-17** cycle (teacher bank expansion + retrain, 2026-10-10)
236
+ **closed it**: **all 12/12 cells ≤ 0.05 on the same frozen holdout** (worst `score/fr`
237
+ 0.0257, `score/it` **0.0014** at accuracy 0.9928), `choice` 0.9248 → 0.9808 and every other
238
+ axis improved or byte-identical. **NFR-C06 met**; the trail is in BACKLOG
239
+ **B-1**/**B-15**/**B-16**/**B-17**.
240
  The CUDA-graph fast path
241
  (`TACHYONE_FAST=1`) gives **2.53×** p50 on English (17.31 → 6.83 ms) and **3.71×** on the
242
  multilingual five-domain path (14.23 → 3.84 ms) — re-measured on the v0.9.0 artifacts
adapter_config.json CHANGED
@@ -35,9 +35,9 @@
35
  "rank_pattern": {},
36
  "revision": null,
37
  "target_modules": [
38
- "Wo",
39
  "Wqkv",
40
- "Wi"
 
41
  ],
42
  "target_parameters": null,
43
  "task_type": null,
 
35
  "rank_pattern": {},
36
  "revision": null,
37
  "target_modules": [
 
38
  "Wqkv",
39
+ "Wi",
40
+ "Wo"
41
  ],
42
  "target_parameters": null,
43
  "task_type": null,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f7969603c1b3a443d413cc98ffdc423f0d35d759b439e6282784cdeaccdfd95d
3
  size 54088096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb946aab6a74a3e5cbf1811c1c4cf85579941d1b56e9c9208da188848852b616
3
  size 54088096
choice_bank_fit.json CHANGED
@@ -1,10 +1,10 @@
1
  {
2
  "arm": "bank",
3
  "config": {
4
- "bank_dir": "checkpoints/multi_tt_il_fit_bank",
5
  "batch_size": 64,
6
  "choice_rank": 128,
7
- "control_dir": "checkpoints/multi_tt_il_fit_ctrl",
8
  "data_path": "data/train_multi_domains.jsonl",
9
  "device": "auto",
10
  "epochs": 8,
@@ -13,7 +13,7 @@
13
  "max_records": null,
14
  "model_id": "jhu-clsp/mmBERT-base",
15
  "seed": 42,
16
- "trunk_adapter": "checkpoints/multi_tt_il",
17
  "val_split": 0.1
18
  },
19
  "domains": [
@@ -26,10 +26,10 @@
26
  "fit": {
27
  "arm": "bank",
28
  "epochs": 8,
29
- "temperature": 0.996136486530304,
30
- "train_loss": 0.0011428351117203106,
31
  "train_records": 15660,
32
- "val_loss": 0.0011507968665770243,
33
  "val_records": 1740
34
  },
35
  "languages": [
@@ -40,5 +40,5 @@
40
  "nl",
41
  "pt"
42
  ],
43
- "trunk_adapter": "checkpoints/multi_tt_il"
44
  }
 
1
  {
2
  "arm": "bank",
3
  "config": {
4
+ "bank_dir": "checkpoints/multi_b17_il_fit_bank",
5
  "batch_size": 64,
6
  "choice_rank": 128,
7
+ "control_dir": "checkpoints/multi_b17_il_fit_ctrl",
8
  "data_path": "data/train_multi_domains.jsonl",
9
  "device": "auto",
10
  "epochs": 8,
 
13
  "max_records": null,
14
  "model_id": "jhu-clsp/mmBERT-base",
15
  "seed": 42,
16
+ "trunk_adapter": "checkpoints/multi_b17_il",
17
  "val_split": 0.1
18
  },
19
  "domains": [
 
26
  "fit": {
27
  "arm": "bank",
28
  "epochs": 8,
29
+ "temperature": 0.9909090399742126,
30
+ "train_loss": 0.013521413194443138,
31
  "train_records": 15660,
32
+ "val_loss": 0.020827124596054517,
33
  "val_records": 1740
34
  },
35
  "languages": [
 
40
  "nl",
41
  "pt"
42
  ],
43
+ "trunk_adapter": "checkpoints/multi_b17_il"
44
  }
choice_head.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:34409891d55ba3c09feac69d694c7e6326855699f869f8f90e9692737592acd4
3
- size 39934565
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c7222ea220d28da546c067b10ab1bf343d5ff26d0c7fe1db08984ea9f21ac76
3
+ size 39897981
confidence_calibration.json CHANGED
The diff for this file is too large to render. See raw diff
 
state_prototypes.json CHANGED
The diff for this file is too large to render. See raw diff
 
temperature_calibration.json CHANGED
@@ -4,134 +4,134 @@
4
  "choice": {
5
  "by_language": {
6
  "de": {
7
- "ece_after": 0.008217,
8
- "ece_before": 0.015348,
9
  "n": 830,
10
- "temperature": 3.0
11
  },
12
  "es": {
13
- "ece_after": 0.034263,
14
- "ece_before": 0.073255,
15
  "n": 840,
16
- "temperature": 6.0
17
  },
18
  "fr": {
19
- "ece_after": 0.003171,
20
- "ece_before": 0.003911,
21
  "n": 830,
22
- "temperature": 2.0
23
  },
24
  "it": {
25
- "ece_after": 0.027815,
26
- "ece_before": 0.063041,
27
  "n": 830,
28
- "temperature": 6.0
29
  },
30
  "nl": {
31
- "ece_after": 0.004955,
32
- "ece_before": 0.010624,
33
  "n": 830,
34
- "temperature": 3.0
35
  },
36
  "pt": {
37
- "ece_after": 0.044754,
38
- "ece_before": 0.076013,
39
  "n": 840,
40
- "temperature": 6.0
41
  }
42
  },
43
- "ece_after": 0.011135,
44
- "ece_before": 0.040317,
45
  "n": 5000,
46
- "temperature": 6.0
47
  },
48
  "noul": {
49
  "by_language": {
50
  "de": {
51
- "ece_after": 0.022377,
52
- "ece_before": 0.242584,
53
  "n": 830,
54
- "temperature": 0.25
55
  },
56
  "es": {
57
- "ece_after": 0.08133,
58
- "ece_before": 0.211321,
59
  "n": 840,
60
- "temperature": 0.25
61
  },
62
  "fr": {
63
- "ece_after": 0.043647,
64
- "ece_before": 0.26635,
65
  "n": 830,
66
- "temperature": 0.15
67
  },
68
  "it": {
69
- "ece_after": 0.027566,
70
- "ece_before": 0.189535,
71
  "n": 830,
72
- "temperature": 0.5
73
  },
74
  "nl": {
75
- "ece_after": 0.046929,
76
- "ece_before": 0.215839,
77
  "n": 830,
78
- "temperature": 0.25
79
  },
80
  "pt": {
81
- "ece_after": 0.060943,
82
- "ece_before": 0.207737,
83
  "n": 840,
84
- "temperature": 0.25
85
  }
86
  },
87
- "ece_after": 0.053913,
88
- "ece_before": 0.215547,
89
  "n": 5000,
90
- "temperature": 0.25
91
  },
92
  "score": {
93
  "by_language": {
94
  "de": {
95
- "ece_after": 0.018422,
96
- "ece_before": 0.42241,
97
  "n": 830,
98
- "temperature": 0.25
99
  },
100
  "es": {
101
- "ece_after": 0.041522,
102
- "ece_before": 0.40273,
103
  "n": 840,
104
  "temperature": 0.25
105
  },
106
  "fr": {
107
- "ece_after": 0.012019,
108
- "ece_before": 0.432135,
109
  "n": 830,
110
  "temperature": 0.25
111
  },
112
  "it": {
113
- "ece_after": 0.049433,
114
- "ece_before": 0.393083,
115
  "n": 830,
116
- "temperature": 0.25
117
  },
118
  "nl": {
119
- "ece_after": 0.021803,
120
- "ece_before": 0.42233,
121
  "n": 830,
122
- "temperature": 0.25
123
  },
124
  "pt": {
125
- "ece_after": 0.011156,
126
- "ece_before": 0.435601,
127
  "n": 840,
128
- "temperature": 0.25
129
  }
130
  },
131
- "ece_after": 0.024324,
132
- "ece_before": 0.415722,
133
  "n": 5000,
134
- "temperature": 0.25
135
  }
136
  },
137
  "warnings": []
 
4
  "choice": {
5
  "by_language": {
6
  "de": {
7
+ "ece_after": 0.013708,
8
+ "ece_before": 0.043495,
9
  "n": 830,
10
+ "temperature": 6.0
11
  },
12
  "es": {
13
+ "ece_after": 0.0,
14
+ "ece_before": 5e-06,
15
  "n": 840,
16
+ "temperature": 0.05
17
  },
18
  "fr": {
19
+ "ece_after": 0.002616,
20
+ "ece_before": 0.0096,
21
  "n": 830,
22
+ "temperature": 5.0
23
  },
24
  "it": {
25
+ "ece_after": 0.00251,
26
+ "ece_before": 0.00355,
27
  "n": 830,
28
+ "temperature": 3.0
29
  },
30
  "nl": {
31
+ "ece_after": 0.00271,
32
+ "ece_before": 0.005693,
33
  "n": 830,
34
+ "temperature": 4.0
35
  },
36
  "pt": {
37
+ "ece_after": 0.002972,
38
+ "ece_before": 0.006214,
39
  "n": 840,
40
+ "temperature": 5.0
41
  }
42
  },
43
+ "ece_after": 0.00348,
44
+ "ece_before": 0.011391,
45
  "n": 5000,
46
+ "temperature": 5.0
47
  },
48
  "noul": {
49
  "by_language": {
50
  "de": {
51
+ "ece_after": 0.001729,
52
+ "ece_before": 0.301935,
53
  "n": 830,
54
+ "temperature": 0.1
55
  },
56
  "es": {
57
+ "ece_after": 8.7e-05,
58
+ "ece_before": 0.303815,
59
  "n": 840,
60
+ "temperature": 0.05
61
  },
62
  "fr": {
63
+ "ece_after": 0.002239,
64
+ "ece_before": 0.299088,
65
  "n": 830,
66
+ "temperature": 0.1
67
  },
68
  "it": {
69
+ "ece_after": 0.005682,
70
+ "ece_before": 0.302041,
71
  "n": 830,
72
+ "temperature": 0.1
73
  },
74
  "nl": {
75
+ "ece_after": 0.006508,
76
+ "ece_before": 0.292156,
77
  "n": 830,
78
+ "temperature": 0.15
79
  },
80
  "pt": {
81
+ "ece_after": 0.000725,
82
+ "ece_before": 0.3015,
83
  "n": 840,
84
+ "temperature": 0.1
85
  }
86
  },
87
+ "ece_after": 0.00323,
88
+ "ece_before": 0.298892,
89
  "n": 5000,
90
+ "temperature": 0.15
91
  },
92
  "score": {
93
  "by_language": {
94
  "de": {
95
+ "ece_after": 0.001757,
96
+ "ece_before": 0.469981,
97
  "n": 830,
98
+ "temperature": 0.15
99
  },
100
  "es": {
101
+ "ece_after": 0.002654,
102
+ "ece_before": 0.444646,
103
  "n": 840,
104
  "temperature": 0.25
105
  },
106
  "fr": {
107
+ "ece_after": 0.002539,
108
+ "ece_before": 0.44168,
109
  "n": 830,
110
  "temperature": 0.25
111
  },
112
  "it": {
113
+ "ece_after": 0.004361,
114
+ "ece_before": 0.466521,
115
  "n": 830,
116
+ "temperature": 0.15
117
  },
118
  "nl": {
119
+ "ece_after": 0.003243,
120
+ "ece_before": 0.466974,
121
  "n": 830,
122
+ "temperature": 0.15
123
  },
124
  "pt": {
125
+ "ece_after": 0.009933,
126
+ "ece_before": 0.457518,
127
  "n": 840,
128
+ "temperature": 0.15
129
  }
130
  },
131
+ "ece_after": 0.0108,
132
+ "ece_before": 0.457823,
133
  "n": 5000,
134
+ "temperature": 0.15
135
  }
136
  },
137
  "warnings": []