All writing

Smart Note · Series

Robot Data

Data sources, collection methods, and data systems for robot learning.

10 articles

Robot Data EP01: Yang Gao and Chuan Wen on Learning Robot Policies from Internet Videos with ATM
Robot Data · EP01

Robot Data EP01: Yang Gao and Chuan Wen on Learning Robot Policies from Internet Videos with ATM

The ATM authors discuss point trajectories, cross-embodiment learning and using human videos to reduce the need for action-labeled robot data.

2024-06-30 · 13 min read
Robot Data, Episode 2: Cheng Chi and Zhenjia Xu on UMI, a Physical API for Robot Training
Robot Data · EP02

Robot Data, Episode 2: Cheng Chi and Zhenjia Xu on UMI, a Physical API for Robot Training

A 2024 conversation with Cheng Chi and Zhenjia Xu about UMI’s handheld hardware, data collection, policy learning and generalization.

2024-07-06 · 16 min read
Robot Data: Xuxin Cheng on Immersive Robot Teleoperation with Open-TeleVision
Robot Data · EP03

Robot Data: Xuxin Cheng on Immersive Robot Teleoperation with Open-TeleVision

Xuxin Cheng discusses stereo vision, active head movement, motion retargeting and learning robot skills from immersive teleoperation.

2024-07-07 · 13 min read
Robot Data, Episode 4: Chen Wang on MimicPlay, DexCap and Learning from Human Motion
Robot Data · EP04

Robot Data, Episode 4: Chen Wang on MimicPlay, DexCap and Learning from Human Motion

The full interview on hierarchical planning, human play, wearable motion capture, retargeting, dexterous control and the differences between DexCap and UMI.

2024-07-14 · 17 min read
Robot Data EP05: Zipeng Fu on HumanPlus and Mobile ALOHA—From Synthetic Data to Real Data, and Combining the Two
Robot Data · EP05

Robot Data EP05: Zipeng Fu on HumanPlus and Mobile ALOHA—From Synthetic Data to Real Data, and Combining the Two

Zipeng Fu discusses how HumanPlus and Mobile ALOHA use simulation, human motion and real-world teleoperation data to train robots.

2024-07-21 · 11 min read
Robot Data, Episode 6: Yecheng Jason Ma on Eureka, DrEureka and LLMs for Simulation and Reinforcement Learning
Robot Data · EP06

Robot Data, Episode 6: Yecheng Jason Ma on Eureka, DrEureka and LLMs for Simulation and Reinforcement Learning

A full conversation on automatic reward design, domain randomization, sim-to-real transfer, reinforcement learning and VIP.

2024-07-30 · 28 min read
Robot Data, Episode 7: Hang Zhao and Ziwen Zhuang on Parkour and Autonomous Driving
Robot Data · EP07

Robot Data, Episode 7: Hang Zhao and Ziwen Zhuang on Parkour and Autonomous Driving

The complete discussion of locomotion, simulation, reinforcement learning, RGB perception and lessons from autonomous-driving data loops.

2024-08-20 · 17 min read
Robot Data, Episode 8: Chuang Gan and Zhou Xian on RoboGen, Generative Simulation and Differentiable Physics (With a Research Review)
Robot Data · EP08

Robot Data, Episode 8: Chuang Gan and Zhou Xian on RoboGen, Generative Simulation and Differentiable Physics (With a Research Review)

A full interview on generating robot demonstrations, learning skills with differentiable simulation, and the research behind key physics platforms.

2024-08-24 · 20 min read
Robot Data, Episode 9: Lirui Wang on Heterogeneous Robot Data and Policy Composition
Robot Data · EP09

Robot Data, Episode 9: Lirui Wang on Heterogeneous Robot Data and Policy Composition

The complete PoCo interview on combining simulation, human video, real robots and tactile information, followed by HPT, fleet learning and a personal reflection.

2024-09-03 · 17 min read
Robot Data, Episode 10: A Full Review of Season 1
Robot Data · EP10

Robot Data, Episode 10: A Full Review of Season 1

Revisiting all nine Season 1 interviews: human videos, simulation, motion capture, teleoperation and the composition of heterogeneous robot data.

2024-09-24 · 19 min read