Document + illustrate the ROI seam-masking detector; re-assert checkpoints
Browse files
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
-
tags: [robotics, surface-roughness, grinding, contrastive-learning, RNC]
|
| 4 |
---
|
| 5 |
|
| 6 |
# grind β visual_grind_grading checkpoints
|
|
@@ -9,6 +9,65 @@ Trained artifacts for the `visual_grind_grading` approach of the
|
|
| 9 |
[grind](https://github.com/BabaYaga840/grind) repo. Gitignored there; hydrate a
|
| 10 |
fresh clone with `./pull_checkpoints.sh`.
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
## Grading model (RNC)
|
| 13 |
`outputs/rnc_grader/rnc_sandpaper_grader.pt` β the RNC sandpaper Ra grader.
|
| 14 |
`torch.load` gives a dict: `Enc` encoder `state_dict` (3-conv CNN, dim 32) +
|
|
@@ -26,6 +85,16 @@ grade = w @ c["grit_classes"].astype("float32") # soft ordinal,
|
|
| 26 |
Validated leave-angle-out rho ~0.93 vs true grit (real, texture-grounded).
|
| 27 |
|
| 28 |
## Other artifacts
|
| 29 |
-
- `outputs/roi_cnn/roi_cnn.pt` β TinyNet ROI CNN weights (grinding ROI, state_dict).
|
| 30 |
- `cache/*.npz` β p27βp38 embedding caches (multisession / SupCon / RNC
|
| 31 |
experiments) for reproducing analyses without recompute.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
tags: [robotics, surface-roughness, grinding, contrastive-learning, RNC, segmentation]
|
| 4 |
---
|
| 5 |
|
| 6 |
# grind β visual_grind_grading checkpoints
|
|
|
|
| 9 |
[grind](https://github.com/BabaYaga840/grind) repo. Gitignored there; hydrate a
|
| 10 |
fresh clone with `./pull_checkpoints.sh`.
|
| 11 |
|
| 12 |
+
Two independent models, answering two different questions about a weld bead being
|
| 13 |
+
ground off a metal plate:
|
| 14 |
+
|
| 15 |
+
| model | question | file |
|
| 16 |
+
|---|---|---|
|
| 17 |
+
| ROI detector | **where** must the robot grind? | `outputs/roi_cnn/roi_cnn.pt` |
|
| 18 |
+
| RNC grader | **how rough** is this surface? | `outputs/rnc_grader/rnc_sandpaper_grader.pt` |
|
| 19 |
+
|
| 20 |
+
## ROI detector (seam masking)
|
| 21 |
+
|
| 22 |
+
`outputs/roi_cnn/roi_cnn.pt` β masks which blocks of the plate are weld seam that
|
| 23 |
+
needs grinding, vs bare plate. A plain `state_dict` for `TinyNet`
|
| 24 |
+
(`scripts/p7_cnn_train.py`): 3 Γ (conv3Γ3 β BN β ReLU) at 16/32/64 channels with 2
|
| 25 |
+
max-pools, global-avg-pool, `Linear(64β1)`, sigmoid. **23,873 parameters.**
|
| 26 |
+
|
| 27 |
+

|
| 28 |
+
|
| 29 |
+
Green β P(seam), red = annotated ROI polygon, blue = plate. Neutral-lit passes
|
| 30 |
+
(top) give a tight dense band; the blue-lit passes are sparser β that lighting
|
| 31 |
+
change is the model's hardest case.
|
| 32 |
+
|
| 33 |
+
Operating config β the detector is **fully convolutional over blocks, not over the
|
| 34 |
+
image**: slide a 24 px context window over the plate and paint the 12 px block at
|
| 35 |
+
its centre.
|
| 36 |
+
|
| 37 |
+
| | |
|
| 38 |
+
|---|---|
|
| 39 |
+
| input | 32Γ32 BGR, `/255`, = the 24 px context window resized |
|
| 40 |
+
| paint block | 12 px |
|
| 41 |
+
| threshold | **0.80** (best F1 0.703, P 0.68 / R 0.72); 0.5 β recall 0.91 |
|
| 42 |
+
| plate bbox | `[500, 412, 713, 519]` |
|
| 43 |
+
|
| 44 |
+
```python
|
| 45 |
+
import torch, torch.nn as nn
|
| 46 |
+
|
| 47 |
+
class TinyNet(nn.Module):
|
| 48 |
+
def __init__(s):
|
| 49 |
+
super().__init__()
|
| 50 |
+
s.f = nn.Sequential(
|
| 51 |
+
nn.Conv2d(3, 16, 3, padding=1), nn.BatchNorm2d(16), nn.ReLU(), nn.MaxPool2d(2),
|
| 52 |
+
nn.Conv2d(16, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),
|
| 53 |
+
nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(),
|
| 54 |
+
nn.AdaptiveAvgPool2d(1))
|
| 55 |
+
s.head = nn.Linear(64, 1)
|
| 56 |
+
def forward(s, x):
|
| 57 |
+
return s.head(s.f(x).flatten(1)).squeeze(1)
|
| 58 |
+
|
| 59 |
+
net = TinyNet()
|
| 60 |
+
net.load_state_dict(torch.load("roi_cnn.pt", map_location="cpu"))
|
| 61 |
+
net.eval()
|
| 62 |
+
prob = torch.sigmoid(net(x)) # x: (N,3,32,32) in [0,1], BGR
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
Leave-one-pass-out (train on 11 passes, test the held-out one): pooled F1 0.632
|
| 66 |
+
@0.5, neutral passes F1 0.701, blue-lit passes F1 0.580. Cross-lighting robustness
|
| 67 |
+
comes from plate-restricted sampling, context windows, and colour/brightness
|
| 68 |
+
jitter augmentation. Full write-up:
|
| 69 |
+
[`ROI_DETECTOR.md`](https://github.com/BabaYaga840/grind/blob/main/approaches/visual_grind_grading/ROI_DETECTOR.md).
|
| 70 |
+
|
| 71 |
## Grading model (RNC)
|
| 72 |
`outputs/rnc_grader/rnc_sandpaper_grader.pt` β the RNC sandpaper Ra grader.
|
| 73 |
`torch.load` gives a dict: `Enc` encoder `state_dict` (3-conv CNN, dim 32) +
|
|
|
|
| 85 |
Validated leave-angle-out rho ~0.93 vs true grit (real, texture-grounded).
|
| 86 |
|
| 87 |
## Other artifacts
|
|
|
|
| 88 |
- `cache/*.npz` β p27βp38 embedding caches (multisession / SupCon / RNC
|
| 89 |
experiments) for reproducing analyses without recompute.
|
| 90 |
+
|
| 91 |
+
Not hosted: `outputs/p7_samples.npz`, the ROI detector's training patches. Rebuild
|
| 92 |
+
with `scripts/p7_build_samples.py` (needs the raw rosbags) if you want to retrain.
|
| 93 |
+
|
| 94 |
+
## Scope note
|
| 95 |
+
|
| 96 |
+
The ROI detector works. The broader hypothesis this approach set out to test β
|
| 97 |
+
that image appearance encodes *cumulative* grinding, so pass index is recoverable
|
| 98 |
+
from a photo of the plate β came out **negative** on this dataset; the strong
|
| 99 |
+
correlations were a lighting/equipment confound. The sandpaper RNC grader above is
|
| 100 |
+
the texture-grounded replacement. See the repo README for that story.
|