Deploying robot policies trained in simulation to real hardware often fails when simulator physics mismatches reality. Vid2Sid introduces a video‑driven system identification pipeline that couples foundation model perception with an optimizer that infers simulator physics discrepancies from paired simulation and real videos, then proposes parameter updates with natural language rationale. Evaluated on both rigid‑body and soft‑body dynamics, Vid2Sid recovers physics parameters with mean relative error under 13 %, outperforming black‑box optimizers that range from 28 %–98 % error.
How I’d pilot this in 10 business days
- Integrate the Vid2Sid calibration pipeline into your existing simulation workflow.
- Collect paired videos around target tasks (e.g., manipulation trajectories).
- Run the optimizer to estimate simulator parameter adjustments and validate them on real hardware.