DPP-v2 PT512 — co-train fine-tuned checkpoints
21-keypoint hand-motion policy: predicts a 16-step x 2-hand x 21-keypoint 3-D displacement flow plus a per-keypoint contact ratio at ~20 Hz, from ego RGB-D object point clouds, the robot's own keypoint state and a per-hand language instruction.
Layout
| folder | what |
|---|---|
ours/<run>/<step>.pt |
flow DiT co-train FT, contact input channel ON (pre-2026-09-22 convention) |
ours_nocontact/<run>/<step>.pt |
same recipe, constant-zero contact input (current design; contact_input=false in the file) |
ours_legacy_ar/<run>/<step>.pt |
the dropped autoregressive variant, kept for the record |
vitra/<emb>_<mix>/<step>/ |
VITRA-VLA-3B baselines (weights.pt, meta.json, config.json, mixture_plan.json) |
groot/<emb>_<mix>/<step>/ |
GR00T N1.7 baselines (safetensors + configs, no optimizer) |
egovla/<emb>_<mix>/<step>/ |
EgoVLA (VILA-3B) baselines |
Run names: <embodiment>_ours_<variant>_<mixture>_<steps>k_gbs64_<n>g[_nocontact]; steps 10000/30000/60000/100000.
| axis | values |
|---|---|
| embodiment | wuji (RB-Y1 + Wuji Hand 2) · inspire (OpenArm + Inspire RH56) |
| variant | flow (rectified-flow DiT, VITRA-style expert) · legacy (deterministic AR) |
| mixture | r0_h100 / r50_h100 / r100_h100 — human 200 episodes fixed, robot 0 / 100 / 200 |
| step | 60000 (matched-budget comparison point) · 100000 (final) |
Training budget: 60 000 / 100 000 steps at global batch 64, fine-tuned from a 100 K-step pre-trained checkpoint. Architecture: context encoder 8L x 768 + action expert 12L x 768, 191 M parameters.
What is in a file
Optimiser and scheduler state are stripped. Each file is a torch.save dict with:
| key | use |
|---|---|
model |
PT512Policy.load_state_dict(..., strict=True) |
variant |
"flow" or "deterministic" — selects the sampling path |
xyz_center, xyz_scale |
(16,2,21,3) de-normalisation: delta = pred * scale + center |
contact_pos_weight |
(2,21) training-time BCE weighting, for reference |
arch_version |
architecture id guard |
contact_input |
false = constant-zero contact input channel (2026-09-22 design, _nocontact runs); absent/true = packed contact fed back (older runs) |
Inference
The policy emits a normalised displacement, not a pose. See
docs/DEPLOY-INFERENCE-GUIDE-2026-09-21.md in the project repo for the full input contract
(nine tensors, observation-time camera anchor frame), the flow-sampling loop, and
de-normalisation. The 21-kp + contact to joint-target adapter is the dpp_v2.deploy package.
Caveats
- The contact head is trained from human data only; robot episodes in the fine-tuning set carry no contact labels. Treat its absolute calibration on a robot with suspicion.
- Keypoints are in MANO / OpenPose-21 order; the hand axis is
[left, right]. - These are research checkpoints from an in-progress study; numbers are not final.