Documentation

This page collects the platform, dataset, policy, protocol, and result details behind the project site, and identifies the retained artefacts and commands behind every number. It documents completed work and does not authorise new training runs or robot trials.

Thesis

Learning Contact Representations from Zero-Cost Proprioceptive Effort Signals for Contact-Rich Visuomotor Imitation Learning. Jaydeepkumar Ranpariya, master's thesis, Friedrich-Alexander-Universität Erlangen-Nürnberg, hosted by the Fraunhofer Institute for Integrated Circuits (IIS), 2026. Supervised by Prof. Michael Philippsen and Dr. Tobias Feigl (FAU) and Alexander Mattick (Fraunhofer IIS).

Overarching question

Can weak, uncalibrated motor-current signals already available on a low-cost robot provide a useful learned contact representation for contact-rich visuomotor imitation learning?

Contributions

• A reproducible characterisation of all six STS3215 current channels as candidate learning inputs: acquisition, distribution, pose dependence, contact discrimination, and temporal behaviour.
• An extended LeRobot-compatible schema and a synchronised 100-episode multimodal peg-in-hole dataset with human-reviewed phase annotations for the analysed subset.
• Controlled ACT implementations for direct current concatenation, training-derived preprocessing, and a learned joint-selective temporal representation.
• A Diffusion Policy vision-and-position system and a learned-current counterpart evaluated under a shared dataset and execution timebase.
• A model-agnostic counterfactual diagnostic that measures how predicted actions change when one modality is replaced while the rest are held fixed.
• A software and verification framework that reproduces the analyses, plots, observation transformations, and evaluated policy settings.

Platform

SO-101 leader and follower arms with six STS3215 bus servos each. Teleoperation runs at 30 Hz with one-to-one joint mapping; the follower wrist roll is limited to 80 encoder counts per step to avoid a firmware overflow. The one-way mapping does not reflect follower current back to the operator, and the operator has no live camera feed.

Effort register

• Datasheet scale 6.5 mA per count; pipelines keep raw counts. Observed task range 0–77 counts, about 0–500 mA.
• Diagnostic read loop over six registers on six joints: 55.3 Hz (±0.1 Hz across poses), 18.1 ms cycle, against the camera-paced 33.3 ms policy period. Policy-rate Nyquist limit 15 Hz.
• Static noise: per-pose standard deviation between 0.02 and 0.49 counts, so variation at rest is mostly integer quantisation.
• Pose dependence: shoulder-lift current 0.1 counts folded to 25.8 counts extended over 20 poses × 3 repetitions.
• Contact rises last roughly 100–300 ms, three to nine samples at 30 Hz. Firmware stall protection pulls current toward 20 counts after about 2 s of sustained stall.

Task geometry

• Final peg: 16.45 mm measured diameter, 70 mm height, 3D-printed PLA, grasped directly by the gripper.
• Hole: IndustRealKit-based block, measured bore 16.8 mm (nominal 16 mm), no chamfer.
• Diametral clearance 0.35 mm, clearance-to-diameter ratio 0.021, four times tighter than the pilot fixture (15.4 mm chamfered peg, 1.4 mm clearance).
• Peg start position fixed and marked about 7 cm from the hole. Vertical peg, grasped from the side.

Channel selection

The gripper load register discriminates contact strongly (d = 1.27 to 19.52) but is coarsely quantised, saturates at 500 counts, and correlates with current at ρ = 0.88 over 51,474 samples, so it was excluded as redundant. Speed, voltage, and temperature are diagnostic only. The selected observation is the six joint positions and six currents; camera input is unchanged. The operating point stays at raw counts with no dynamics model, torque calibration, or force reconstruction.

Dataset

Schema

All recorded datasets

Total unique: 200 episodes and 197,253 frames (the 100-episode set re-uses the two vertical batches). The training set for every matched comparison is peg-tight-vertical-100: 100 episodes, 99,831 frames, 30 Hz, mean duration 33.3 s, four sessions, two cameras, all successful insertions.

Current distribution and normalisation

• 44.9 % of all current samples are exactly zero; per channel 35–54 %. Up to 94 % of wrist-roll samples are at most one count.
• Shoulder-pan skewness 11.3 with excess kurtosis near 400 (median 0, q99 = 3, maximum 52 counts).
• Plain mean–std maps free-space zeros to an offset of −0.5 to −0.9 and maxima to +6 to +55 standard deviations. The selected transform winsorises each channel at the per-joint training q99.5 before mean–std scaling; caps and statistics are fitted on training demonstrations only.
• Four training views: act-v (6-dim state), act-m (12), act-m-norm (12, winsorised), merged (18, diagnostics only). Episodes, frame indices, images, timestamps, and action targets are identical across views.

Recording protocol

episode_time_s = 45, reset_time_s = 15, fps = 30, video = true. Temperature checked every ten episodes with a pause if any joint exceeds 60 °C. One operator. Phase annotations: 32 human-reviewed rows considered, one non-monotonic row excluded, 31 valid rows retained as calibration metadata, never as training labels. LeRobot fork JDRanpariya/lerobot, branch feat/proprio-logging.

Policies

ACT-V and DP-V receive the six joint positions; ACT-M and DP-M receive positions and currents (12 dimensions). ACT runs at 30 Hz. The recovered Diffusion Policy runs at a native 10 Hz with stride-3 data. ACT uses mean–std normalisation for state and action; DP uses mean–std for images and min–max for state and action.

ACT configuration

Diffusion Policy configuration

All variants

The historical ACT-M-CNN artefact was most likely trained on raw-count history windows but deployed on already-normalised windows; this is recorded as a provenance-supported implementation limitation. Implementation-stage extensions (residual refiners, auxiliary future-current and contact-state objectives, visual curriculum) are unevaluated and contribute no result rows.

Evaluation protocol

• All fixed-task policies train on the same 100 demonstrations with seed 1000. Hardware performance never selects among seeds, so repeated trials estimate deployment and session variability, not training variance.
• Success: the policy autonomously grasps, transports, and seats the peg flush before the limit. Partial insertion is a failure. Failure categories: wrong_approach, no_grasp, dropped, missed, partial, timeout, other.
• Interleaving: alternating blocks of six trials, five blocks per policy, 30 trials per variant and 60 per pair. Rollout timeout 45 s, inter-trial reset 15 s.
• Thermal gating: every servo at or below 50 °C before a block starts; a sustained 65 °C over five samples forces a pause until all joints cool to 50 °C.
• Statistics: success proportion with a Wilson 95 % interval; two-sided Fisher exact test per pair; Holm correction when more than one pairwise p-value is reported; predeclared 10-point practical threshold.
• Sessions are reported separately and never pooled.

Results

Matched comparisons

Block successes in the final DP session: DP-V 3, 4, 2, 0, 0; DP-M 5, 2, 2, 2, 1. The unchanged ACT-V checkpoint scored 17/30 in the normalised session and 7/30 in the temporal session, which prohibits ranking ACT-M-norm (15/30) against the learned encoder (12/30) across sessions.

Task-difficulty ablation (ACT-V)

Removing the chamfer, not the clearance reduction alone, produced the first significant drop. The vertical grasp and the side camera each change geometry and visibility at once, so the decreases cannot be assigned to a single factor. With 100 demonstrations the second camera introduces wrong-approach failures that are absent from the one-camera condition.

Historical ACT fusion ablation (30 trials each)

All 30 FiLM failures occur before insertion (18 wrong_approach, 12 no_grasp). Both the CNN and FiLM rows are implementation-limited: valid for the deployed artefacts, but they cannot cleanly reject either architecture.

Screens and pilots (not confirmatory)

• Bounded ACT feature-conditioning screen, one block of five trials per policy: ACT-V control 3/5; warm-start concat 2/5; bounded FiLM from scratch 1/5; bounded FiLM warm start 1/5; hard-latched FiLM warm start 3/5. No checkpoint promoted.
• Behaviour-preserving DP screen, five trials per model: DP-V control 4/5; DP-M concat 0/5; instantaneous dual-FiLM 3/5; jointCNN K8 dual-FiLM 4/5; hard-latched 1/5. Model identity is confounded with run order.
• Recovered DP-V confirmatory session: 17/30 (Wilson [39.2, 72.6]). Direct DP-M-norm and K3/K8 encoders: 0/5 each, descriptive only.
• One-step DP replanning: neither arm begins purposeful motion; rejected as a controller failure, not current-modality evidence.

Reproduction record

Values derived from data are recomputed from retained CSV, Parquet, JSON, and annotation files. Configuration values are checked against policy code, resolved commands, checkpoint configuration, and evaluation manifests. Literature claims are checked against same-key source files in the reference archive.

Deterministic checks

python3 experiments/phase-1-characterisation/scripts/reproduce_chapter4_numbers.py
python3 research/writing/verify_chapter5_dataset.py
python3 research/writing/verify_chapter6_method.py
python3 research/writing/verify_chapter7_evaluation.py
python3 research/writing/audit_final_manuscript.py

Wilson and Newcombe intervals shown on the site are recomputed at build time from the success counts in src/data/results.json and match the manuscript values. The dataset, checkpoints, trial records, and scripts accompany the thesis in the project archive; public release is not assumed. Large recordings and checkpoint tensors are not embedded in this website.

Media provenance

The four clips are derived from retained thesis recordings. MP4 derivatives use H.264, yuv420p, and a fast-start index; source frame rates are kept and no audio exists. The full record with source paths, SHA-256 digests, trial UUIDs, and the exact FFmpeg commands is kept in website/media-provenance.md.

The ACT-V and ACT-M clips occupy the same rotated block and trial slot but are separate physical rollouts; they illustrate the aggregate result and are not synchronised paired evidence. The demonstration clip's original annotation label reads "zero-shot (direct)", which describes a direct-entry teleoperated demonstration, not a learned zero-shot policy.