In 2024, Smart Note published ten paper-based interviews explaining different kinds of embodied data; see EP10: A Summary of Robot Data Season One. In 2025, we followed with ten interviews about the human data used in robot training; see Human Data—The “Key” to Robot Data. I have always regarded data as a vantage point from which to observe embodied AI. How deeply a company understands data determines the ceiling of its embodied models.
Unfortunately, few embodied-model companies in China have developed genuinely original thinking about data. We therefore need a data company that understands it better than model companies do, because that could substantially raise the level of the whole industry.
ModalityNet (https://modalitynet.com) is a data initiative that I believe can accelerate embodied AI and move the whole industry forward. It aims to build the world's largest human-centered, all-modality dataset for embodied intelligence.
With a series of questions about data, we invited Dr. Ruoli Dai, founder of Noitom Robotics and a creator of ModalityNet, to an in-person interview. More than 100 friends joined me in hearing his explanation of the project. I believe the following conversation contains authentic thinking about embodied data that the whole industry should see. Set aside a quiet hour for this interview about a dataset I strongly recommend.
Original article outline
Contents
👇
1. About ModalityNet
The scale of ModalityNet's data
2. MOV: High-precision human interaction data in large spaces
How MOV data is collected
MOV's modalities compared with AMASS and LAFAN1
MOV's scale compared with AMASS and LAFAN1
3. OM: High-precision data for fine human hand manipulation
How OM data is collected
The scale of OM data
What can such precise hand data be used for?
4. The relationship between MOV and OM
5. ITW: Human egocentric manipulation data from uncontrolled environments
How ITW data is collected
6. How much embodied data is needed today?
7. How can data companies respond to changing model paradigms?
8. Reconstructing the inputs through which humans perceive the world
9. The World Compiler
10. How can we improve data learnability?
11. How can we validate improved learnability?
1. About ModalityNet
When deciding what data we should collect—or what a relatively pure, neutral third-party data company like ours should supply—the property I value most is transferability across embodiments.
That determines whether data can be reused by different robotics companies and across their different robot models. It makes economic sense. Data that supplies training material for the whole industry and transfers across embodiments becomes shared data infrastructure. Its value lies in repeated use throughout the industry, rather than service to a single company.
Imagine a dataset that thirty robotics companies can use. Suppose producing it costs ten yuan. As its producer, I do not need to recover the entire cost from one customer. Serving twenty or thirty companies can cover collection, annotation, quality control, ongoing iteration, and subsequent R&D.
As more companies use the same category of data, its marginal cost falls, and the whole industry can obtain high-quality training data more cheaply. That is the greatest value of cross-embodiment data: reuse at scale spreads data costs across the industry.
A third-party supplier should therefore primarily provide cross-embodiment data. Customers should produce data that cannot transfer. A company using a particular robot should collect data usable only by its own robot, such as embodiment-specific real-robot teleoperation or UMI data tied to a particular gripper's capabilities. These datasets naturally serve that embodiment or gripper, so its manufacturer should handle them rather than a third party.
ModalityNet is our newly launched dataset, accessible at modalitynet.com. At the time of this interview, the website has three categories, all usable across embodiments: MOV, multimodal large-space interaction data; OM, multimodal dexterous manipulation data; and ITW, multimodal egocentric data collected in the wild. ModalityNet is the public dataset we have released across those three categories.

The scale of ModalityNet's data
We keep the publicly released data at roughly one-twentieth of what we hold. For example, if we have 200,000 hours, approximately 10,000 hours would be public. Barring major changes, we intend to maintain that ratio. The remaining nineteen-twentieths underpin our business model and are sold; we will keep the public twentieth open. [Editor's clarification: the interview uses “open-source” for this release; actual use is governed by ModalityNet's Open Research License and its conditions.]
ModalityNet currently has several thousand hours of data, with several thousand individual recordings directly previewable online. Several thousand additional hours are available upon request, after establishing a collaboration or completing a simple application process.
We collaborate extensively with academia, including HKUST, the University of Hong Kong, Tsinghua, Southern University of Science and Technology, the University of Science and Technology of China, HKUST Guangzhou, CUHK Shenzhen, UC Berkeley, Stanford, UC San Diego, and the University of Southern California, among many others.
We work with many excellent researchers and support them with data after discussing their needs in person. Some larger research groups have a dozen or more PhD students collaborating with us and directly requesting substantial datasets. Sometimes we even provide customized support for particularly scarce data.
We built the dataset around our understanding of algorithmic needs. The categories correspond to whole-body control or motion foundation models (MOV), fine tabletop manipulation models (OM), and visual generalization and pretraining (ITW). Its creation was both proactive and reactive: driven by our understanding of training, and by demands from customers.
2. MOV: Multimodal Large-Space Interaction Data
The first category serves whole-body control and loco-manipulation. Its backbone consists of human motion and poses, object trajectories, and third-person vision. Manipulation tasks also include hand poses. We call it MOV: Motion, Object, and Vision.
It is inherently multimodal, combining human pose, object trajectories, object meshes—geometry as well as trajectories—and vision from third-person, first-person, and other viewpoints. Such data can transfer across embodiments in training many robot models.
NVIDIA's recently released SONIC and Kimodo, for example, use substantial quantities of this kind of data. See An Interview with Zhengyi Luo: The SONIC Whole-Body Control Framework. The GAE work discussed in Human Data EP06—MoCap + Video: GAE, and Figure AI's Helix whole-body control work, also make extensive use of MOV-type data.

Example of HiPHI-MOV data
How MOV data is collected
MOV requires a large capture space. We have a data-collection facility of almost 10,000 square meters in Longhua, Shenzhen, containing many MOV capture areas.
People can move freely and perform tasks together. For example, four people might carry a table. I might want to train four robots to collaborate, or two robots with two people, or one person leading three robots, to move a large round table into another room. They may need to pass through a revolving door and stand the table upright. That requires many strategies and complex coordination.
How do we collect that data? First, the space must be large. Second, several people must interact simultaneously. We need continuous tracking and a mesh for objects such as the table. Lower-precision human manipulation, including coarser hand poses, must also be captured completely and supplemented with vision. MOV therefore corresponds fundamentally to large-space data collection.

Example of a MOV capture setup
MOV's modalities compared with AMASS and LAFAN1
This first category is fundamentally motion-centered data for whole-body control.
I want to emphasize a distinction. LAFAN1, AMASS, and the dataset Bones Studio recently produced for NVIDIA GEAR Lab are, in the sense I mean here, single-modality datasets centered on body poses. When we capture those poses, we also record environmental vision, object trajectories, and other information. [Editor's clarification: “single-modality” contrasts motion-centered data with synchronized interaction sensing; it does not imply that these datasets lack auxiliary annotations such as text.]
Example of MOV data in use
MOV's scale compared with AMASS and LAFAN1
Another major difference is scale. As I recall, LAFAN1 contains a single-digit number of hours, or perhaps a dozen or so. AMASS, from Michael Black's team at the Max Planck Institute in Germany, is an excellent dataset that I used as a student. The version I am discussing contains roughly seventy-plus hours, with forty-plus hours that I consider usable.
We now hold more than 8,000 hours of this category of data—roughly two orders of magnitude more, or about a hundredfold increase. That is data we already possess.
For research in whole-body control, loco-manipulation, and whole-body VLAs, this dataset is therefore already usable.

Comparison of MOV dataset scale
(Pink: MOV; yellow: LAFAN1; blue: AMASS; green: MotionX++.)
3. OM: Multimodal Dexterous Manipulation Data
ModalityNet's second category targets fine manipulation, especially high-precision hand motion capture. We call it OM—Omni Manipulation, or broadly applicable dexterous manipulation. You can think of it as an extremely scaled-down version of MOV. [Editor's clarification: the current official catalog expands HiPHI-OM as “Omni-Modality”; “Omni Manipulation” is the interview's explanation of its purpose.]
Unlike MOV's large capture areas, OM reduces a tracking space of 100, 200, or even 300 square meters to tabletop scale, while imposing extremely demanding precision requirements.
It captures finger poses at submillimeter precision, along with whole-body poses, first-person RGB and RGB-D, third-person multiview RGB and RGB-D, and even spatial audio.
Why include audio? Many manipulation tasks do not depend entirely on vision. Suppose I am speaking to this audience and place an object on a table. For me, the table's existence is a visual prior; what really confirms completion is the sound of placing the object down. Sound is therefore important information too.
Touch also matters. Suppose I make a motion and take the object back without using my eyes. Prior knowledge, force sensation, touch, and proprioception let me complete it. We capture this information in OM as well.

Example of HiPHI-OM data
How OM data is collected
In OM capture areas, you will see many small booths densely equipped with sensors and cameras. They capture human manipulation with extremely high precision, including in-hand manipulation, tactile signals, and motion-control information. We aim to capture the entire process. This is ModalityNet's second category.
We combine optical and inertial motion capture, with additional sources such as electromagnetic sensing. Touch is particularly important for fine hand manipulation, so we also include the tactile modality.
Because the tasks take place around a fixed workspace, we additionally capture first-person vision and depth, third-person vision, multiview third-person vision, and multiview depth.
We also capture modalities whose future usefulness is not yet clear, such as temperature and audio. I include them because I worry they may become valuable later.
The precision is very high, and the modality coverage is even more complete than in the first category.
For whole-body control data, suppose I slowly push a chair forward. Given the hand pose and chair trajectory, it is relatively easy to infer the relationship between hand and chair.
For fine in-hand manipulation, however, hand poses and a bottle's state alone make it difficult to estimate the subtle contact relationship accurately. Tactile information is needed to properly describe the contact between person and object during fine manipulation.
The scale of OM data
This dataset is smaller because it is particularly difficult and expensive to collect. We currently hold a few thousand hours. Under the ratio I described, dividing by twenty leaves a public release of a few hundred hours—approximately that scale.
What can such precise hand data be used for?
First, it can train a manipulation foundation model, helping a model understand comprehensively how humans manipulate objects. It is a manipulation-level counterpart to a whole-body motion foundation model such as Kimodo.
This particularly interests me personally. I am working with several leading researchers on it.
I expect related projects to appear from the second half of this year through early next year. They will explore manipulation foundation models at the tabletop level. That is OM's first role.
The second role is alignment to a robot embodiment. The original data comes from humans, including body and hand poses, touch, and object trajectories. We can fully align it to a specific robot—for example, a Unitree platform equipped with a Sharpa Wave dexterous hand. We call that embodiment alignment.
It is a complicated process involving forward and inverse dynamics, mapping, retargeting, reinforcement learning, extensive simulation training, and finally sim-to-real transfer. Once the entire pipeline is complete, the data is essentially very close to teleoperation data because it is fully aligned with that robot.
Yet it can go beyond teleoperation. Many actions are difficult or impossible with the teleoperation systems I am referring to: solving a Rubik's Cube, precise manipulation heavily dependent on force and tactile feedback, or fine throwing and catching actions. Humans can perform them naturally.
We can therefore first collect human data and then align those actions completely to the robot. Once aligned, the data matches the embodiment, and in synthetic space we can directly obtain corresponding motion-control information, generating motor-level control data.
OM's second role is thus to substitute for teleoperation data after human–robot embodiment alignment, while covering fine manipulation tasks that teleoperation itself cannot accomplish in those settings.
Example of OM data in use
4. The Relationship Between MOV and OM
(Is MOV high-precision human motion capture for humanoids, while OM is high-precision hand motion capture for dexterous hands?)
That is both right and not quite right. Fundamentally, I see them as the same thing. MOV fully digitizes human interaction in large spaces; OM fully digitizes fine human manipulation.
The equipment choices, modality composition, and even the capture software belong to the same system. One workspace is twenty by twenty meters, the other three by three meters. But the smaller one's construction cost is already quite close to that of the larger one.
Motion capture is only one modality; I must emphasize that. Motion or pose is not, in my view, the most important modality for robots. Vision still comes first. I would rank vision first, touch second, and pose third, including both body and hand pose. Robots also need many others, such as object trajectories and terrain.
If I could retain only one modality, it would be vision. With two, I would keep vision and touch. Whole-body pose would be third, and hand pose fourth.
I have good reasons for that ordering. Vision can estimate much hand-pose information, and even indirectly infer some tactile information. The reverse is not true: without vision, it is simply gone. Without touch, when vision is occluded, hand and body poses cannot recover the missing tactile information.
So your interpretation is partly right. We did start through the human- and hand-pose technologies we know best. But objectively assessing what robots need, I would not even call pose the most important modality.
5. ITW: Multimodal Egocentric Data Collected in the Wild
ModalityNet's third category centers on egocentric vision collected in the wild. ITW stands for “in the wild,” meaning uncontrolled environments. Egocentric vision is a person's first-person view—for example, from a worn camera or a phone fixed to the head. This category is captured with head-mounted equipment in real settings.
Why emphasize egocentric vision? Data collected indoors, in studios, or in laboratories using numerous outside-in sensors, such as optical motion-capture cameras and UWB sensors, is factory or studio data. Those are controlled environments.
True in-the-wild data comes from uncontrolled real environments: actual eldercare facilities, logistics warehouses, or flexible manufacturing lines. Egocentric vision is the backbone modality, and we also try to incorporate pose and touch.
Ideally, we would reconstruct as many of the modalities involved in human perception as possible: touch, body pose, hand information, egocentric vision, and ground-truth-level depth—preferably with LiDAR on the shoulder too. Eventually our capture operator would be covered in devices like a Christmas tree.
This closely resembles the perspective on human data described in our earlier conversation with Danfei. See An Interview with Danfei Xu: Human Data Is Robot Data in Disguise.
But such a setup cannot truly scale in the wild to 100,000 or a million hours. This year's industry demand for in-the-wild data has already reached that range, so we must make trade-offs and sacrifice some modalities.
ITW capture setup
How ITW data is collected
(How does ITW differ from the first two categories, and from the egocentric vision data currently available on the market?)
ITW uses equipment we designed ourselves, containing multiple cameras: two fisheye cameras, an RGB camera, another camera configurable as high-precision RGB or RGB-D, and four grayscale cameras. The grayscale cameras primarily support visual SLAM, head tracking, and hand tracking. The device is also very open to integrating third-party equipment for other modalities at any time.
Our concept of an egocentric device therefore goes beyond putting a GoPro on someone's head. Once a device is worn there, it should cover as much of the desired information and modality set as possible.
It has storage, computing, and data transmission capabilities, with long battery life: a charge generally supports a full day's continuous collection. Its field of view, or FOV, almost completely simulates human vision.
How wide do you think human FOV is? With the eyes fixed, I describe it as roughly 110–120 degrees. Most current VR headsets are designed around 105 degrees; some reach 120, but edge distortion becomes substantial there, so around 110 remains mainstream. Our fisheye camera has a 210-degree FOV. [Editor's clarification: these are the interview's approximate comparisons; binocular overlap, total peripheral visual field, and a headset's display FOV are different quantities.]
Why wider than human eyes? Because eyes can rotate. If I look straight ahead without moving them, a hand moving far enough to the side disappears from view. Eye movement extends what we can perceive; I describe the overall accessible field as exceeding 180 degrees and reaching approximately 220, including regions toward the back of the ears.
Robots and this device have no rotating eyeballs, so we use a 210-degree wide-angle field to simulate the overall view available when human eyes turn.
The surrounding grayscale cameras all have global shutters. Compared with ordinary rolling shutters, they provide more precise temporal alignment and fewer image distortions or tearing within a frame. We find the grayscale configuration less susceptible to natural-light interference in this application, making it well suited to hand and head tracking. This is a typical egocentric device for us.
Does wearing that device complete the collection setup? Yes and no.
In most scenarios requiring very large scale—over 100,000 hours, for example—or crowdsourcing, this is essentially the only equipment we can ask people to wear. It is difficult to require crowdsourced participants to add motion-capture equipment, tactile gloves, or other complicated devices.
In more controlled situations, our own collection fleet can wear additional equipment, such as motion-capture devices and tactile gloves, to capture more modalities together. Some leading robotics teams now need precisely this “Ego Vision Plus,” rather than egocentric vision alone.
Previously, wearing a phone or GoPro could collect basic data that met many needs. Such data now struggles to fully satisfy training requirements. It is heavily commoditized, and even where there is demand, prices have fallen substantially.
Many teams already collect it in India, French-speaking Canada, Mexico, Southeast Asia, and provinces across eastern and central China. Downstream customers generally want monocular or stereo video, with different prices. The real barrier lies in Ego Vision Plus, which has more demanding capture requirements.
6. How Much Embodied Data Is Needed Today?
My strongest impression from last year to this year is how astonishingly fast customer demand has grown.
Before October last year, conversations were still in units of a few thousand hours.
Near the end of the year, Generalist released work publicly reporting a dataset of roughly 270,000 hours. It was mainly handheld-gripper data, not pure egocentric vision, with only a small portion including that viewpoint. From that day onward, customers began discussions at 300,000 hours.
Early this year, teams such as Generalist and Sunday used scales of 500,000 hours or even tens of millions of data samples to investigate scaling laws. Demand rapidly jumped to 500,000 hours, which became the new starting point.
In recent months, people have begun discussing a million hours. One overseas company even told us, “Prepare for demand for ten million hours next year.”
That far exceeded my expectations, though I am not necessarily convinced.
In my assessment, even autonomous driving has not used data on that scale so far. My concern is less whether customers can afford the data than whether their compute budgets can support both data and training costs, and whether they can actually handle such a volume.
I still think the ten-million-hour figure contains some exaggeration, but I regard a million hours as real demand. We are expanding production capacity this year around that target.
7. How Can Data Companies Respond to Changing Model Paradigms?
Abstractly, the question is whether constant changes in downstream models are good or bad for us. I think volatility is good. In business competition, convergence is more frightening than change.
The dangerous outcome would be a handful of big customers, completely fixed requirements, transparent prices, and an ever-growing number of suppliers. Everyone would compete only on price, with margins reduced to barely surviving rather than making money.
Continuing iteration and new technical approaches downstream instead create more opportunities for data companies.
The real question is not how to follow every model change, but what these models are learning. I strongly agree with the view that this wave of embodied AI, humanoid robotics, and data-industry development is fundamentally a revolution in biomimetics.
Robot bodies increasingly resemble human anatomy, models increasingly approach how the human brain and cerebellum learn, and data sources increasingly rely on real human interaction and movement.
We should therefore focus less on what a particular model needs today and more on the fundamental information humans use to complete a task.
Whatever the architecture, I think models are essentially learning two things: rules and human priors.
We organize capture around the human, rather than a specific robot or model architecture. I want all the physical information a person relies on during a task to be digitized, at a precision at least matching what human perception can detect.
For a simple example, I just adjusted my seated posture. It is a small action, but it uses many different kinds of information.
First, vision supplies priors, so it does not need particularly high precision here. I only need to know roughly where the audience is.
Second, proprioception tells me about my body and hand poses. In the interview's simplified account, the cerebellum performs this role for humans; for robot training data, we digitize it using motion capture. Proprioception indicates my current pose and how to adjust toward the target. Touch at the buttocks, legs, and soles also participates. [Editor's clarification: proprioception involves peripheral receptors and multiple neural circuits, rather than the cerebellum alone.]
To transfer this capability to a robot, I must capture the corresponding information: body poses, environmental information, terrain, the chair's geometry, tool trajectories, touch, and proprioception. Together these modalities comprise the information humans rely on for this movement.
My response to changing model paradigms is therefore simple. Whether the future uses VLAs, world action models, or another architecture, the information humans rely on to perform tasks remains the same.
I want to digitize all that valuable human information at a reasonable cost. If these modalities are captured completely, I believe the data will retain its value as model paradigms change. Over the long term, it will not become obsolete.
8. Reconstructing the Inputs Through Which Humans Perceive the World
(Danfei Xu made a similar point in An Interview with Danfei Xu: Human Data Is Robot Data in Disguise. The central question is how humans use their senses to interact with the world, and how robots can reproduce those human inputs.)
Exactly. We center on human interaction, whether large-space movement or fine manipulation, and completely digitize every genuinely valuable modality. I see this as distilling the physical world and human capabilities themselves.
Consider a more specific example. Why did we ultimately choose flexible tactile sensors? They do not have the highest precision. The reason is simple: human touch itself does not have such high absolute accuracy.
In my view, almost all human perception is relative rather than absolute: pose, brightness, pain, weight, force, and even color are perceived through relative changes. Stare at a red screen for a long time and other things afterward appear greenish. That illustrates the relative nature of vision.
When choosing sensors, if I must trade off absolute accuracy, range, robustness, and stability, I often sacrifice absolute accuracy first. That differs from the approach of many robotics teams.
Many people first examine precision metrics such as scale factor, cross-coupling error, and RMS performance. I examine them too. But when they conflict with resolution, range, stability, or robustness, I prioritize the latter. Human perception lacks exceptionally high absolute accuracy yet has tremendous range, robustness, stability, and self-repair capability. I think those are the qualities robots should learn.
9. The World Compiler
I wrote a blog specifically about this: https://noitomrobotics.com/world-compiler/. There are two relevant posts available through ModalityNet. Our chief scientist, Dr. Lei Han, wrote the first; I wrote the second, titled “World Compiler.”

I intended it as a manifesto, addressed both to our internal team and to the industry. The internet keeps a record, and I wanted to put our views on record in advance. Even if some prove wrong years later, I am not afraid of being contradicted. I want people to know that these were not retrospective conclusions: our group explicitly proposed them long before.
As I said, Noitom Robotics' data business essentially distills information from the physical world and human capabilities. I see that distillation as two stages.
First, record the world through data collection. The resulting raw data has little learnability by itself: it is unstructured, containing basic sensor and visual signals. Of course, a black-box large model may learn many capabilities in latent space on its own.
Second, compile the raw data into information models can learn directly. Our years of experience suggest that first “compiling” it into more structured information makes training significantly more efficient and improves data ROI. Structuring and abstracting data is also a form of compression: it extracts the most important information and focuses the model's learning.
I borrow “compile” from software development. Python and C++ programs are written in high-level languages, relatively close to natural language. Machines cannot directly execute that source; translation by a compiler or language runtime makes execution possible. [Editor's clarification: this is a software analogy; Python commonly uses bytecode and an interpreter, whereas C++ is typically compiled to machine code.]
Raw data is like source code: it is not necessarily in the best form for model learning. A compilation layer can turn it into more structured, learnable data and make training more efficient.
For example, an egocentric video initially contains only vision. If we additionally estimate hand poses, depth, tactile information, and perhaps infer whole-body poses from what is available, data that was difficult to learn from becomes easier. The process continually increases structure and learnability. [Editor's clarification: inferred modalities are estimates, not automatically equivalent to directly measured ground truth.]
Improving raw-data structure and learnability is an important future direction for the industry. Someone must do it. If data companies do not, the data teams inside model companies eventually will.
I am confident about our position because many customers' data teams have only five or six people and cannot continually invest the resources needed for a complete compilation pipeline. We have more than a hundred people focused on this task, allowing us to specialize much more deeply.
Extracting more learnable information from data is valuable. Almost every customer now requests it, including text and semantic annotations: sometimes per second or per frame, sometimes segmented by scene or task. These are all ways to improve learnability.
If a data company merely buys equipment, hires people, collects data, and delivers it unchanged, fully competitive pricing ultimately amounts to equipment depreciation plus labor costs.
Competition drives pricing back toward costs: returns correspond to what you put in. Learnability is different. It is a “black box,” and the part in which data suppliers and customers create real value.
If we develop more capabilities within it—improving structure and extracting more learnable information—the data can command a premium. That builds a real competitive barrier for the data business while giving model companies higher-quality data and more efficient training. Both sides benefit.

10. How Can We Improve Data Learnability?
(We have discussed learnability throughout. Could you give concrete examples of improving existing data? How does each of Noitom's three categories increase learnability?)
The learnability of the three categories is quite different.
MOV and OM are highly learnable because they are already strongly structured and contain substantial ground-truth-level information: touch, body poses, and other measured modalities. Models can learn from them directly.
Even so, they need additional information such as scene, process, and semantic annotations. Post-trained multimodal large models can now automatically produce high-quality semantic annotations. Several Silicon Valley teams specialize in this.
They charge to automatically annotate large quantities of egocentric data. An additional few dollars per hour, across 100,000 hours, becomes a business worth hundreds of thousands of dollars. This is already an independent data-services market.
The third category, largely egocentric vision, is much less learnable. It is effectively a strict subset of the first two all-modality categories, retaining vision and only sparse other modalities. Much information is inherently missing, lowering learnability.
The three datasets support one another. Our full-modality, multidimensional accumulation lets us postprocess the third category much better than the industry norm. Our depth or hand-pose estimates might be more accurate: where others' hand-pose errors are around two centimeters, we can achieve subcentimeter results. We can even estimate leg poses when others have not filmed the legs. These are the capabilities I find truly interesting.
11. How Can We Validate Improved Learnability?
(The data is ultimately used for model learning, but we do not know exactly which information a model learns from. How do we determine what information is valuable and validate that?)
I would push the question deeper: how do we and our customers verify that data is good, or that improving learnability has actually created value? That is the data industry's greatest challenge.
Have you noticed how unfair data transactions can be to suppliers? This is a market in which value is judged after the fact.
Suppose I sell 100,000 hours at 200 yuan per hour: twenty million yuan in total. The customer trains a model, and four or five months—or half a year—later it performs poorly, the loss does not fall, and success is only 40%. Disputes inevitably follow. The customer says, “Your data is bad.” I might reply, “Your model team lacks the ability.” These disagreements are almost unavoidable.
Such transactions share a feature: margins tend not to be very high. Customers manage risk by pressing the price down in advance. They reason, “I must pay now, but will not know the outcome for months, so I have to lower the price.”
That naturally affects data companies' gross margins. I think the best approach is to prove the data's value along the customer's technical route before delivery.
This is why more than twenty of our eighty or ninety full-time employees conduct model research. People find it surprising that a data company needs so many researchers, but that is the reason.
Their work has three parts: perception and interaction algorithms; data augmentation, modality completion, and postprocessing; and model-based validation.
Our researchers use the same technical routes as customers. They train VLAs, world action models, motion foundation models, and teleoperation foundation models, but at a much smaller scale. We are validating data, not training a commercially deployable model.
For example, we randomly sample one-tenth or one-hundredth of the complete dataset and check whether loss continues to converge. We also examine scaling: take one percent of a 5,000-hour dataset and one percent of a 50,000-hour dataset, and see whether greater scale and better distributions yield better training benefits.
That is the training return we want to verify.
(So what you mean by “World Compiler” is that physical AI needs more than raw data: it needs data models can actually understand and learn from. A data company's role is to extract those learnable properties?)
Yes. Put even more simply: while we cannot capture every modality and dimension, can we complete what is missing? That completion is a process of interpretation, and the problem the World Compiler really seeks to solve.

Editor's Note
“An embodied-data company essentially distills the physical world and human capabilities. Its value lies in compiling signals collected by sensors from the physical world, and signals representing human capabilities, into data models can learn from directly.” That is my concise interpretation of our guest's central argument, and the problem this dataset aims to address.
References
1. Noitom Robotics website
2. ModalityNet
https://noitomrobotics.com/art-of-modalities/
3. World Compiler
https://noitomrobotics.com/world-compiler/
Bones Studio