Retrieval oracle on the motherboard dataset: for each in-contact tactile frame of the held-out test episodes (motherboard_0510_episode_005, motherboard_0510_episode_006), the nearest-neighbour in-contact frame from other episodes is retrieved by gel-contact-pose distance in the object frame (position + 0.227 mm/deg × (tilt + twist)), and scored against the ground truth. If contact geometry fully determined the tactile image this oracle would be near-perfect; the residual is the irreducible texture detail (marker layout, gel state, pressure/shear history).
| method | psnr | ssim | gradcorr | gradcorr_diff |
|---|---|---|---|---|
| copy reference (episode blank frame) | 37.75 ±1.98 | 0.939 ±0.02 | 0.669 ±0.10 | 0.000 ±0.00 |
| random in-contact retrieval | 32.30 ±0.80 | 0.888 ±0.01 | 0.275 ±0.04 | 0.072 ±0.03 |
| frozen last-context frame (windows, n=1827) | 38.25 ±3.29 | 0.940 ±0.03 | 0.696 ±0.12 | 0.404 ±0.21 |
| oracle NN (K=1), cross-episode | 32.59 ±2.32 | 0.882 ±0.03 | 0.265 ±0.16 | 0.110 ±0.12 |
| oracle best-of-5, cross-episode | 33.75 ±2.16 | 0.897 ±0.03 | 0.333 ±0.17 | 0.170 ±0.13 |
| oracle best-of-20, cross-episode | 34.73 ±1.82 | 0.908 ±0.02 | 0.387 ±0.17 | 0.216 ±0.13 |
| oracle NN (K=1), within-episode allowed (±2 s excluded) | 35.14 ±3.50 | 0.908 ±0.04 | 0.482 ±0.22 | 0.268 ±0.20 |
| oracle best-of-20, within-episode allowed | 37.33 ±2.76 | 0.931 ±0.02 | 0.640 ±0.15 | 0.417 ±0.16 |
NN distance median 5.9 mm (p90 11.7); pose-distance vs PSNR: Pearson 0.061, Spearman 0.073 over 9750 pairs.
| method | psnr | ssim | gradcorr | gradcorr_diff |
|---|---|---|---|---|
| copy reference (episode blank frame) | 39.26 ±2.45 | 0.953 ±0.01 | 0.769 ±0.11 | 0.000 ±0.00 |
| random in-contact retrieval | 32.75 ±1.17 | 0.908 ±0.01 | 0.527 ±0.08 | 0.062 ±0.04 |
| frozen last-context frame (windows, n=1462) | 38.18 ±3.16 | 0.941 ±0.02 | 0.719 ±0.14 | 0.253 ±0.21 |
| oracle NN (K=1), cross-episode | 33.22 ±4.35 | 0.887 ±0.06 | 0.510 ±0.20 | 0.092 ±0.11 |
| oracle best-of-5, cross-episode | 35.77 ±4.06 | 0.916 ±0.05 | 0.621 ±0.18 | 0.168 ±0.11 |
| oracle best-of-20, cross-episode | 37.43 ±3.17 | 0.934 ±0.03 | 0.701 ±0.13 | 0.238 ±0.08 |
| oracle NN (K=1), within-episode allowed (±2 s excluded) | 36.37 ±2.96 | 0.926 ±0.03 | 0.662 ±0.14 | 0.167 ±0.13 |
| oracle best-of-20, within-episode allowed | 38.57 ±2.35 | 0.943 ±0.02 | 0.756 ±0.10 | 0.342 ±0.10 |
NN distance median 6.5 mm (p90 10.2); pose-distance vs PSNR: Pearson 0.092, Spearman 0.099 over 9750 pairs.
Frames are 192×256 RGB, fps6. gradcorr_diff = Sobel gradient correlation after subtracting the per-episode no-contact reference (static gel pattern removed). Left-sensor DB excludes 5 episodes from 05-19 (physical gel swap; different marker layout). Consecutive fps6 frames autocorrelate, so ±sd overstates independence.
| distance | left | right |
|---|---|---|
| position only (w_ang=0) | 33.49 / 0.247 | 34.07 / 0.099 |
| default (w_ang=0.227 mm/deg) | 32.59 / 0.110 | 33.22 / 0.092 |
| strong angle (w_ang=1) | 31.76 / 0.215 | 33.61 / 0.115 |
| default + contact-area term (w_area=20) | 32.89 / 0.247 | 34.28 / 0.114 |
| reference number | value | note |
|---|---|---|
| world model (mot_df_vt_actionflow, long_test 16s) | tl 30.5 / tr 31.4 dB | whole-clip PSNR incl. 8 GT-context frames, 256x256 decode - convention differs from the per-frame oracle numbers |
| Wan2.2-VAE round-trip texture ceiling | grad-energy ratio ~0.86 vs raw | eval_texture_ceiling.py; any latent-space model is additionally capped by this |
Binned mean ±sd PSNR of database frames against the query GT, by contact-pose distance. The left end is the oracle regime; the flat right side is the unconditional (random-retrieval) floor.
Examples are deliberately stratified from the closest to the farthest pose match (plus the single best and worst PSNR queries), ordered best-first — the later panels are the failure modes, not typical retrievals. Panels marked CARRIED PART are frames where the gel grips a component held above the board; board-relative contact pose cannot determine the tactile image there, and they are included to show that limit. Compare the two right-most (reference-subtracted) columns rather than the raw frames: matches usually reproduce the same imprint at lower or higher pressure.
Middle view with projected gel pose (cyan dot, red normal, green u-axis) | registered-mesh height map under the gel (26×20 mm window, turbo colormap) | GT tactile | NN1 tactile frame retrieved from the other episodes by contact pose (source episode, frame and pose distance in the caption). Episode 001 is a training episode; its own frames are excluded from the retrieval pool.
