Anyone working on humanoid motion control will be familiar with Xue Bin (Jason) Peng. Over the past year, high-quality demonstrations of human-like humanoid movement have appeared across many platforms. Much of their technical foundation can be traced to his pioneering work, including DeepMimic, AMP (Adversarial Motion Priors) and ASE (Adversarial Skill Embeddings). These methods substantially lowered the barrier to motion imitation for humanoids and helped establish the foundations for flexible, natural movement. The discussion retains its April 2025 context.
At the time of this interview, Xue Bin (Jason) Peng is an assistant professor at Simon Fraser University (SFU) and a research scientist at NVIDIA. His work lies at the intersection of computer graphics and machine learning, particularly reinforcement learning for motion control of simulated characters.
His academic path is also noteworthy. His advisers—Michiel van de Panne, Pieter Abbeel and Sergey Levine—are among the most influential leading professors in character animation and robotic reinforcement learning.
During his master's studies at the University of British Columbia (UBC), Jason worked with graphics researcher Michiel van de Panne. In addition to his foundational contributions to character animation, van de Panne supervised the master's research of Andrej Karpathy, formerly Tesla's AI director, and the PhD of Stelian Coros, who leads ETH Zurich's Computational Robotics Lab.
Jason then pursued his PhD at the University of California, Berkeley, advised by Sergey Levine and Pieter Abbeel, building a strong foundation in reinforcement learning and robot intelligence.
In this interview, we discuss his research journey and several foundational contributions to humanoid whole-body motion control, along with his ideas for general-purpose controllers and recommendations for robot hardware. I hope it will be useful to people developing humanoid software and hardware.
Special thanks to Dr. Dun Yang and Dr. Qiang Zhang for helping prepare the interview and correct the transcript. I also thank Dr. Zhicheng Wang, Dr. Haojie Shi and Dr. Ziwen Zhuang for contributing several important questions. The podcast episode can be found by searching for ReSpark on Xiaoyuzhou.
Original article outline
Contents
↓
From graphics to robotics
1. The influence of his advisers
2. From simulated characters to real robots
3. Challenges in robotics
4. Approaching sim-to-real transfer
Foundational projects
5. DeepMimic
6. AMP
7. ASE
8. MaskedMimic
Reinforcement learning and imitation learning
9. Comparing their strengths and weaknesses for locomotion
10. Bottlenecks for reinforcement learning in manipulation
Robot foundation models
11. Perspectives on VLA models
12. Has industry surpassed academia in motion control?
13. Future research: general-purpose controllers
14. Building a general-purpose controller
15. Is a latent space necessary?
16. Perspectives on mixture-of-experts architectures
17. Recommendations for humanoid hardware
From graphics to robotics
1. How did Michiel van de Panne, one of the most influential professors in character animation, and Pieter Abbeel, a pioneer of robotic reinforcement learning, shape your research and thinking?
Michiel was my adviser during my undergraduate and master's studies at UBC. Honestly, without him, I might not be doing this research at all. I was interested in computer graphics and wanted to try research in the field. Michiel worked on character animation at UBC, so I contacted him about participating in animation research. That was how I entered the area.
I was not thinking about robotics then; I simply wanted to work in graphics. Animation looked interesting, but machine learning had not yet become central to the work we were considering. Much of it relied on traditional kinematic methods or hand-designed controllers.
Later, DeepMind published Continuous Control with Deep Reinforcement Learning. After reading it, we began studying reinforcement learning and deep reinforcement learning more closely as a way to train controllers for virtual characters. That was a turning point that gradually brought me into machine learning and its connection with animation. Editorial note: the original calls the method deep Q-learning; the cited paper introduces Deep Deterministic Policy Gradient (DDPG), an actor-critic method extending ideas from deep Q-learning to continuous actions.
During my PhD at Berkeley, I began working with Sergey Levine and Pieter Abbeel and gradually applied these techniques to controlling real robots, rather than only characters in virtual worlds.
Michiel, and later Sergey and Pieter, had a profound influence on my career. I feel very fortunate to have worked with them. Without their guidance, I might now be doing something entirely different.
2. Your research shifted from learning skills for virtual characters in graphics to reward-based training of real robots—a move from simulation to reality. What is the biggest difference between those directions?
When I started working with real robots, I was surprised by how difficult things that seem obvious in simulation become in reality.
State estimation, for example, was much harder than I had expected. In simulation, we can have full observability: positions, velocities and even the agent's complete internal state are directly available.
On a real robot, even obtaining something as basic as linear velocity can be difficult. You need additional sensors and estimation algorithms, and the final estimate may still be noisy and inaccurate.
Partial observability was therefore one of the most striking challenges in my transition from simulation to real robotics. It remains a difficult problem today.
What is partial observability?
An environment is partially observable when an agent cannot access all information about the system's complete state. Its decisions are based on only part of the environment, with observations that may be incomplete, uncertain or noisy. A robot moving through a room with only a forward-facing camera, for example, cannot see behind itself or to its sides. It may need to remember where it came from or estimate the environment through a sequence of actions.
A fully observable environment
In chess, every piece and its position are visible on the board. The game state is available, allowing decisions based on complete information, though finding the optimal move still requires computation.
A partially observable environment
Imagine driving through fog while wearing dark glasses. You can see only a few meters ahead, rather than the whole route. You must infer the present state and choose actions using limited observations, past experience and internal memory.
3. What has left the deepest impression on you when working with real robots?
State estimation certainly stands out, but another major issue is that real-world dynamics are non-stationary: they change over time. In a typical simulation, the dynamics are largely fixed. That makes experiments easy to reproduce. Once a controller works, it can generally be run repeatedly with very consistent behavior.
A real robot's dynamics keep changing. A controller may work well today and behave differently later. Motors heat up after prolonged use, for example, changing the system dynamics. A controller that worked ten minutes ago might no longer be effective when you run it again.
4. How do you address challenges in sim-to-real transfer?
The main challenges include partial observability, system uncertainty and changing real-robot dynamics. Our central tool is domain randomization. In most of our studies, we train controllers in simulation and then deploy them directly on real robots.
To reduce the reality gap, we randomly perturb physical parameters and observation conditions during training, including mass, friction, latency and sensor noise. This encourages stronger, more adaptive policies that generalize better to uncertainty in reality. Domain randomization remains our main tool for sim-to-real transfer.
Foundational projects
Could you give us a systematic overview of your main research directions and projects? What is the central idea behind each, and which important problems do they address?
5. DeepMimic
I can begin with DeepMimic, an early project. Its basic idea is simple: combine deep reinforcement learning with motion tracking. We wanted simulated characters to imitate a wide range of human movement skills for animation and virtual environments.
DeepMimic
Before DeepMimic, many animation controllers borrowed approaches from robotics, including trajectory optimization, optimal control and finite-state machines. Their biggest problem was a lack of generality and substantial manual design. Each new skill required designing a controller from scratch: one for walking, another for jumping, and much more time and effort for flips or complex acrobatics. This was not scalable over the long term, so much earlier research focused on very few specific behaviors.
With DeepMimic, we wanted a general framework capable of reproducing any human movement skill. We used deep reinforcement learning to train neural-network controllers. Neural networks are powerful function approximators that can represent flexible, complex policies. Instead of manually designing a finite-state machine for every skill, we can use a shared network architecture to adapt to all skills.
What is a finite-state machine?
A finite-state machine, or FSM, is a classical model in computer science and AI for representing transitions between a finite set of states. The system occupies one state at a time and changes state in response to inputs.
Motion tracking is the other key part of DeepMimic. Once we have a neural-network controller, how do we train it to perform different skills? Conventionally, each skill might require its own reward function—one for walking and another for running—which is also difficult to scale.
We use a shared reward mechanism: tracking a reference motion. Give the controller a target motion, such as motion-capture data, and train it to imitate the sequence frame by frame. We found that this simple tracking reward, combined with policy-gradient reinforcement learning, can teach many skills without manually designing a new controller or imitation reward for each one.
Rather than designing separate control logic for every skill, we provide different reference motions and optimize the same training objective to learn the corresponding policies. That is the basic idea of DeepMimic.
6. AMP
An important project after DeepMimic was AMP, or Adversarial Motion Priors. DeepMimic provides a general framework for training controllers by tracking reference motions frame by frame. That is effective when a character should reproduce a particular motion precisely. Many impressive humanoid demonstrations today are built on this kind of tracking technique.
A key limitation is that controllers trained this way are very rigid. They often just play the same motion sequence over and over, cannot deviate from the prescribed trajectory, and cannot adapt well to new tasks. [Editorial note: this describes the limitation motivating AMP, rather than every capability of DeepMimic. The original DeepMimic work also combines imitation with task objectives and demonstrates recovery and multi-skill behavior.]
AMP
In AMP, we wanted a more flexible imitation objective that would not constrain the controller to frame-by-frame tracking. The idea is to replace trajectory tracking with distribution matching. Instead of reproducing one precise trajectory, the controller generates behavior whose motion style matches the dataset overall.
We use adversarial imitation learning, similar to a generative adversarial network, or GAN. A discriminator learns to distinguish motion from the reference dataset from motion generated by a simulated character. The controller simultaneously learns to fool the discriminator by producing behavior that resembles the reference data.
This offers more flexibility. The controller does not need to follow one sequence frame by frame. It can stitch together or combine motion fragments, generalize to behaviors not explicitly present in the dataset, and pursue task objectives that the demonstrations do not directly contain. AMP therefore provides a stronger, more adaptive framework for motion imitation.
Some people reproducing AMP report that their policy learns only transitions between individual frames, leaving the character stepping in place instead of moving. Have you encountered this? Is it a limitation of AMP itself?
Yes, that can happen, depending on the design choices. A policy may get trapped in a local optimum, switching back and forth between one or two frames so that the character appears stuck and does not move.
It can often be addressed by expanding the discriminator's temporal window. Instead of showing it only two frames, show it a sequence of perhaps ten. Observing a longer period makes unnatural behavior such as getting trapped in a single transition easier to recognize, encouraging smoother and more realistic dynamic movement.
7. ASE
Many methods for learning robot movement skills use a latent space to represent features, but that space is not intuitively interpretable to people. How do you control the safety of data in it, preventing unusual latent inputs from causing abnormal policy behavior?
This is an important question. In a subsequent project, we introduced ASE—Adversarial Skill Embeddings—to address this kind of problem.
Many models, especially those using variational autoencoders, or VAEs, have holes in their latent space: some codes decode into behavior that looks nothing like the training data. We do not have a perfect solution, but ASE uses an approach resembling GAN training to reduce the problem.
In ASE, we treat the controller as a decoder. It maps a latent code into a motion sequence that should resemble real reference motions. A discriminator judges whether the generated behavior matches the reference style, much like training a GAN.
An important advantage of adversarial training is that it encourages every point in the latent space, as far as possible to map to plausible behavior. This can reduce the likelihood of abnormal regions and lower the risk of policy errors.
We cannot guarantee that strange behavior will never occur. Some latent codes may still produce unnatural motions. But in our experiments, adversarially trained models such as ASE are generally more robust and stable than traditional methods.
8. MaskedMimic
How large is MaskedMimic's sim-to-real gap?
There is still a substantial gap between MaskedMimic and deployment on real robots. In simulation, we can train controllers to reproduce thousands of motion clips. Most real-robot systems are still at what I call the DeepMimic stage: a controller often performs only one skill or one motion clip.
MaskedMimic
Animation systems can now support thousands of skills in one controller, but reliably transferring such high-capacity controllers to real robots is not yet something we can do.
Sim-to-real transfer is the central issue. Even a well-trained simulated controller often needs substantial manual tuning on the real robot before it works. With thousands or tens of thousands of skills, debugging and validating each one individually in reality becomes prohibitively expensive. That is one of the largest current bottlenecks.
Reinforcement learning and imitation learning
9. What are the strengths and weaknesses of imitation learning and reinforcement learning for movement skills? Which has greater long-term potential?
I am not sure they can be compared directly because they are essentially orthogonal paradigms. Imitation learning can be implemented through supervised learning or reinforcement learning, depending on the available data. Reinforcement learning can implement imitation, but it can also train a controller directly from rewards without demonstrations. They are not strictly opposing approaches.
If we insist on comparing imitation learning with pure reinforcement learning that uses no demonstrations, we can consider the following trade-offs.
The advantage of pure RL is that no demonstration data is required. You define an objective, perhaps a human-designed reward function, and train a controller to optimize it. The agent may discover better behaviors or strategies that people have not considered. Systems such as AlphaGo illustrate RL's ability to discover solutions beyond established human play. Editorial note: the original AlphaGo also learned from human games; AlphaGo Zero and AlphaZero are clearer examples of the demonstration-free learning discussed here.
The disadvantage is that reward design can be complicated. To guide an agent toward a particular behavior, you may need a carefully designed function; otherwise it may learn something different from what you intended. Those are some of RL's strengths and weaknesses.
Imitation learning largely reverses the trade-off. Demonstrations make it easier to guide behavior because the training objective is to reproduce what appears in the data.
That makes it easier to obtain human-like behavior or movement that matches our expectations.
But collecting high-quality demonstrations can be difficult. For some tasks, few people can perform them well, making suitable data hard to obtain. The controller's behavior can also be more constrained: it often learns what humans have already done and has less ability to discover new strategies. In that sense, imitation may limit exploration and creativity.
10. What are the bottlenecks for RL in manipulation?
One reason RL is less widely used in manipulation than in locomotion may be that easier opportunities remain. Behavior cloning and supervised learning already work well. For manipulation, it is relatively straightforward to build a teleoperation system, collect many human demonstrations and train a controller with supervised learning. That often gives good results.
With legged locomotion, high-quality robot demonstrations are much harder to collect, so we were almost forced to use reinforcement learning from the outset. Most manipulation controllers today are still trained with supervised learning and behavior cloning, but I think manipulation will eventually follow a similar path.
I think vision-language-action models, or VLAs, will ultimately follow the development pattern of large language models. LLMs usually begin with predictive training on large amounts of human-produced data—supervision from the data itself in pretraining. To improve performance on a particular task or specialize the model, we then use fine-tuning or task-directed reinforcement learning.
Manipulation models may take a similar route. At present, we can train with supervised learning on large quantities of teleoperation data. To reach the best performance on a specific manipulation task, we may then need RL fine-tuning to optimize the model further.
Robot foundation models
11. More people are exploring large models such as VLAs for robotics. What role do you see for large-scale architectures in learning robot skills?
I think large multimodal models are very promising. We have observed very interesting emergent properties: complex behaviors that arise from these models.
The main bottleneck, in my view, is inference speed. Slow inference makes it difficult to use them for real-time low-level control. It resembles the distinction between slow and fast thinking. Many VLAs may serve as high-level planners running at a lower frequency, with their plans executed by smaller, faster and more responsive low-level controllers. The multimodal models are promising, but particularly for low-level control, we still need good controllers that turn high-level instructions into concrete actions.
A relatively slow VLA can work for slower, less dynamic tasks such as simple pick-and-place manipulation. Agile whole-body motion, however, requires much higher-frequency control. At present, our computational resources are not sufficient to make these large multimodal models meet that requirement.
12. In locomotion control, has industry surpassed academia in some respects?
Yes. In some respects, industry has become better at producing highly agile, human-like robot behavior. Manufacturers understand their robots' underlying systems deeply and know how to extract the best performance from a particular machine. Academics like us often lack that insight because we buy robots rather than build them. Industry therefore has a clear advantage in demonstrations of strikingly agile humanoid motion.
Academia also has strengths, especially in developing more general control models. Instead of reproducing one behavior, we are moving toward control foundation models capable of many human skills. Academia may be slightly ahead in exploring these general models. Their motion quality, however, may still fall short of the smoothness and naturalness of industrial demonstrations.
13. Which future research directions interest you, or seem especially promising?
I am interested in more general control models. Rather than designing a controller for one skill, we want models that reproduce many human movement skills. MaskedMimic and work from CMU such as PHC show promise. Important questions remain: how can we ensure that the controller generates natural-looking behavior, and how can we transfer or reuse it effectively for new tasks? Today, reusable controllers are often limited to simple tasks such as basic navigation. We want stronger general models that can be reused for more complex tasks.
14. What should a future general-purpose controller look like? What can larger control systems learn from generative models?
I think future general control models will fundamentally be generative models. Instead of traditional control modules, they will resemble models that generate images or text, except that they generate motor commands. We will see more generative models serving as the core architecture of control systems.
The next question is how to train these generative controllers in a scalable way on large datasets. If the data includes action labels, supervised learning remains among the most scalable approaches across many domains. Without explicit action information, we need reinforcement learning. In our experiments, motion-tracking-based RL has worked best for training on large motion datasets.
As you mentioned, algorithms such as AMP and other adversarial imitation methods can suffer from mode collapse when scaled: the model produces only a few kinds of behavior and loses diversity. We find that using motion tracking as the objective substantially reduces this problem. It resembles the difference between supervised and adversarial learning, such as GANs.
In recent work such as MaskedMimic, we therefore returned to tracking-based training to develop generative control models.
15. Do we need a latent space, perhaps built with a Gaussian distribution or something like a Fourier transform? Is that structure necessary when designing a controller?
I do not think a latent space is absolutely necessary. It is one mechanism for representing complex distributions in a generative model, but strong generative models can be built without a separately sampled latent-code space. Large language models, for example, are powerful generative models that do not explicitly depend on that kind of latent space. A similar approach may be possible for control. This does not mean LLMs lack internal hidden representations.
Many leading generative models we use do employ some form of latent variables. Diffusion models are an example and are among the strongest generative approaches at present. A latent space may therefore help, but it is not indispensable. What we need is a sufficiently flexible and powerful architecture that captures the complex, multimodal distribution of behavior. Editorial clarification: noise variables in diffusion and a learned, compressed latent space are different concepts; not every diffusion model requires the latter.
16. Will a general-purpose controller use some form of mixture of experts?
I am generally less inclined toward a mixture of experts in which each expert is trained separately. If controllers are trained independently for different skills and combined afterward, it is difficult to ensure that they cooperate well.
Smooth, stable transitions between experts are not guaranteed, making the whole system fragile and limiting control performance. Large models such as language models do use MoE layers, but that is different from simply joining independent models. Those layers are part of a single architecture trained end to end.
I find an integrated MoE architecture—with expert layers inside a jointly trained model—more promising than combining separate experts afterward. As a general research principle, I prefer end-to-end training to training modules independently and then trying to assemble them.
17. What recommendations do you have for current humanoid hardware?
Since much of my work relies on sim-to-real transfer, my main request to hardware designers is to make robots easier to simulate.
Specifically, first, motor behavior should approximate ideal proportional-derivative, or PD, control as closely as possible. Second, the kinematic structure should be easy to reproduce in a physics simulator. That minimizes the sim-to-real gap. Motor dynamics are among the largest sources of mismatch we encounter in practice.
What is PD control?
Proportional-derivative control is widely used in robotics, automatic control and physics simulation. It calculates a control output from the current position error and its rate of change—the velocity error—to drive the system toward a target state. In simple terms, the proportional component drives the correction, while the derivative component provides damping and stability.
When transferring between simulation and a real robot, motors that closely approximate an ideal PD controller reduce the difference between simulated and actual behavior, making control easier.
References
Jason Peng's homepage
DeepMimic
DeepMimic: https://xbpeng.github.io/projects/DeepMimic/index.html AMP: https://xbpeng.github.io/projects/AMP/index.html ASE: https://xbpeng.github.io/projects/ASE/index.html MaskedMimic: https://xbpeng.github.io/projects/MaskedMimic/index.html