Many readers will probably have clicked this headline with real excitement. If you did not, you may not yet be familiar with the progress in 3D vision over the past two years. In this interview, we speak with Jianyuan Wang, first author of VGGT, the CVPR 2025 Best Paper, and VGGT-Ω, a CVPR 2026 Best Paper Finalist.
At the time of this interview, Jianyuan is pursuing a joint PhD with Oxford's Visual Geometry Group and Meta AI. His research asks how a model can perceive the three-dimensional world using images or video alone, without additional sensors. In other words, he wants models to understand and perceive 3D space from visual information alone.
This conversation reminded me of a discussion with a founder two years ago. Today's embodied models build on an open-source base model: previously a vision-language model (VLM), and now a visual geometry model (VGM). So what determines progress in embodied models: the base model, or the company building on top of it? What would the arrival of a hypothetical future “GPT-9.8” mean for today's embodied AI startups?
Again, I strongly recommend this interview. Every paragraph deserves a careful reading; the insights are in the details.
Original article outline
Contents
👇
Part I · 3D Reconstruction
1. What is 3D reconstruction?
2. Why is 3D reconstruction important?
3. Will reconstruction capabilities become part of large multimodal models?
4. Why have robotics and large models not embraced 3D information more fully?
5. Will such a foundation model have to come from a large-model company?
Part II · The VGGT Series
6. Introducing the VGGT series
7. Training data for VGGT
8. What counts as high-quality data?
9. Synthetic data
10. Is 3D data from wearable devices usable?
11. Other questions about VGGT's data
12. Data for VGGT-Ω
13. Is data the biggest bottleneck in 3D research?
Part III · Robotics
14. How can robotics researchers use VGGT?
15. Why does robotics rely so heavily on 2D rather than 3D?
16. Future research directions
Part I · 3D Reconstruction
1. What is 3D reconstruction?
This field has been studied for a very long time. Around the 1980s and 1990s, as computing power developed, researchers began asking a natural question: how can we recover three-dimensional information from two-dimensional information?
We feel that we live in a three-dimensional world. But if we set touch aside and consider how we obtain information, we mainly rely on vision, which essentially receives two-dimensional images or video streams.
The question for early researchers was therefore how to recover the structure of the 3D world from these 2D observations.
One classic early approach was stereo vision. Many readers will have encountered stereo cameras. Put simply, when two cameras observe the same scene at the same time, with a small baseline between them—a slight displacement in position—we can use triangulation to recover the depth of a point from the corresponding pixel positions in the two images.
In a sense, this is a special case of 3D reconstruction, because the relative positions of the stereo cameras are known and fixed. Researchers wanted to handle more general situations in which the relationships between camera positions were themselves unknown. The problem then becomes recovering both the depths of scene points in 3D space and the cameras' positions and orientations.
Roughly between 1995 and 2005, researchers increasingly found that they could solve these problems using a pipeline called structure from motion, or SfM. People in robotics may be more familiar with SLAM. In a certain sense, SLAM can be viewed as a special setting or subset of SfM. [Editor's clarification: this is an analogy to visual SLAM; SLAM also includes online localization and mapping, and can use nonvisual sensors.]
SfM takes a collection of images depicting approximately the same scene and reconstructs camera parameters and scene depth. In the visual SLAM setting being discussed here, the images additionally have a temporal order: they form a continuous video stream. This simplifies aspects of the problem and can make the result more stable and reliable.
So, during the period from 1995 to 2005, the field's main concern was how to formulate and solve this problem.
From about 2005 to 2014, the emphasis shifted toward engineering improvements: making reconstruction faster, larger in scale, and more robust. For example, researchers wanted to process 100,000 images, or eight, ten, or even more hours of video.
Then, from 2014 to 2022, deep neural networks began entering the field. At that time, the prevailing view was that reconstruction already had mature mathematical formulations: triangulation, fundamental-matrix decomposition, and so on all had clear mathematical expressions. The focus was therefore not on having a network perform reconstruction directly, but on using deep learning to improve individual stages of the traditional pipeline.
Take ORB-SLAM as an example. An early step is to find correspondences between multiple images. Researchers investigated how neural networks could produce better feature representations and identify more accurate correspondences in feature maps, improving the reconstruction. That was broadly the mainstream direction between 2014 and 2022.
From 2022 onward, the field began embracing deep learning much more comprehensively, whether by choice or through the direction in which the field was moving.
Representative work includes DUSt3R and, later, VGGT and VGGT-Ω. Their central idea is to stop using neural networks merely to find better correspondences and instead turn the entire reconstruction problem into an end-to-end deep learning problem.
(VGGT's reconstruction on the far left looks visibly better than the two comparisons.)
In other words, it becomes a fully black-box process. Some people like that and some do not, but I personally like the black-box concept. We can now feed images into a trained Transformer and have it directly output the various kinds of 3D information we want.
I think this is also why 3D reconstruction has suddenly attracted renewed attention over the past two or three years.
The first reason is that, if the approach can use a training paradigm similar to those in 2D vision and language models, it has a chance to benefit from scaling laws.
As computing resources, data volume, and training investment increase, the model can improve steadily and consistently.
That is a very attractive property for companies. Businesses are more willing to invest in technical approaches that can deliver dependable returns.
The second reason is that, compared with traditional reconstruction methods, these fully learned approaches are beginning to handle dynamic scenes.
An important idea here is that a dynamic object can be thought of as a combination of many tiny rigid-body motions. Traditional reconstruction frameworks struggle with this because multiple rigid motions occur simultaneously, making it difficult to recover the true structure. But these problems are now beginning to be solvable.
For example, demos from some recent projects can already recover the spatial relationships between a moving person and the surrounding environment. Many previously intractable problems are becoming tractable.
(A VGGT-Ω demo that beautifully illustrates dynamic reconstruction.)
The third point is the one I personally find most compelling. In 2D vision, we are accustomed to encoding an image into a global representation and using it for classification or other downstream tasks. In geometry, at least before 2022, we did not really have an equivalent.
Now we can turn an input video sequence into a set of geometry-aware representations or tokens. These tokens can encode the sequence's main geometric structure and motion information.
Such representations support many new downstream applications and open up interesting research directions.
I see these as the main changes brought by the wave of fully learned 3D reconstruction from 2022 to 2026.
2. Why is 3D reconstruction important?
I would say that 3D reconstruction underpins 3D and 4D understanding—what we might call spatial understanding. To borrow an expression from an older generation of researchers, it is the “crown jewel” of geometry. If we can solve reconstruction well, much of the information we want afterward becomes directly available.
This is why reconstruction has remained important for so many years. Looking at the history of CVPR and related research, academia has consistently paid close attention to reconstruction regardless of changes in technical approach. It occupies a distinctive, foundational position in the field.
In my view, its importance has grown rather than diminished today.
Consider an analogy with the development of 2D deep learning. Researchers often first pretrained a model, such as ResNet, on an image-classification task like ImageNet, then transferred it to object detection, image segmentation, and other tasks. Classification served as a pretraining task that supplied general features for many downstream applications.
I think reconstruction is gradually assuming a similar role in 3D. The features learned by a trained reconstruction model can transfer to many other 3D tasks. In some of our own work, for example, we have used reconstruction features for novel view synthesis. Similar features can also support surface estimation and usually provide substantial benefits.
From this perspective, reconstruction can be a proxy task for spatial understanding. More importantly, it is a scalable proxy task: the relevant data is relatively easy to obtain, without especially expensive annotation.
That is another important reason for the attention it receives.
Using concepts from 2D vision, we can think of reconstruction as a foundational task that, to some extent, combines aspects of image classification and segmentation. It learns both global scene semantics and spatial structure, supplying general capabilities for many downstream tasks.
3. Will reconstruction capabilities become part of large multimodal models?
Architecturally, many VLMs today have a language branch and a visual branch. On the visual side, many use something like CLIP or DINO to turn images into a set of tokens.
Those tokens generally lean more toward semantic information. My feeling is that, if our approach is fortunate enough, it might play a similar role in the future: an additional branch in an existing model, or an extra set of tokens supplied to it.
Our model could then provide geometry-aware and spatially aware tokens to VLMs, vision-language-action models (VLAs), and other kinds of models.
That is a direction I would particularly like to see—the kind of foundational role you mentioned.
4. Why have robotics and large models not embraced 3D information more fully?
I think we can divide this question into two parts.
You asked why 3D capabilities have not really entered robotics yet. I have a slightly different view: the main reason is that the demand is not yet strong enough.
Many people in today's most active areas of robotics are focused on manipulation: a robot arm picking and sorting objects on a table, for example. My intuitive sense is that these tasks are relatively simple conceptually. That does not mean they are easy to accomplish; rather, they do not depend so heavily on 3D understanding. Often, optimizing for a particular task using only 2D data can already produce very good results.
I do not think robotics has reached the point where 3D information is indispensable. Spatial information certainly helps, but it is not yet a necessary condition for solving these tasks.
My expectation is that the real need for 3D capabilities may come in another two or three years. As attention shifts from simpler manipulation tasks to open-world, general-purpose robotics, 3D and spatial understanding will reveal their importance and become essential.
For now, we are mostly seeing meaningful early explorations. People are investigating ways to fuse different kinds of information, but it may take another couple of years for these capabilities to really shine. The industry first needs to solve today's manipulation tasks reasonably well before its attention naturally moves to the next set of problems.
I have discussed this with quite a few students, and have made similar points in recent CVPR talks.
Imagine a future model—let us call it “GPT-9.8”—with strong control capabilities. Give it any robot platform: a humanoid, a robot arm, or some other device. I expect that, at some point in the future, perhaps not far away, such a model could read the manual and understand the device's structure and controls in just over ten seconds, then quickly learn to operate it through one or two minutes of real interaction with the environment.
For example, the robot might take two steps, fall once, run, slide, or jump. Through these test-time interactions, it might learn to control a robot it has never encountered in only a few minutes, in what I describe as a zero-shot setting. [Editor's clarification: this imagined setting includes interaction and adaptation at test time; “zero-shot” here means no prior training specifically for that robot.]
That scenario is also why I think robot models need spatial awareness.
I hope future robot models will no longer require large amounts of specially collected robot data or depend on teleoperation devices and embodied-data collection pipelines. Instead, during pretraining, they would already be multimodal models that understand images, text, and 3D space together.
Then, given any robot interface, they could rapidly understand and operate the robot like an experienced operator, without additional training or dedicated adaptation.
I do not think that era is very far away.
5. Will such a foundation model have to come from a large-model company?
Regarding competition between large-model companies and robotics startups, I personally think the companies building large models are the most likely to produce the eventual general-purpose robot model.
Of course, many robotics startups may emerge along the way, developing advantages in data, hardware, or particular applications. They might even be acquired by large-model companies to supplement their data and hardware capabilities. But when it comes to who will ultimately build the robot model we envision, I still favor the large-model companies.
In my conception, a truly capable future robot model would essentially be a foundation model with an enormous number of parameters. It would need to understand language, vision, and space, as well as have stronger reasoning capabilities than any of today's models.
I think that requires a particular organizational DNA. At present, only a handful of leading large-model companies can continually train and iterate on models at that level.
Many robotics startups may have considerable strengths in hardware, systems integration, or specific applications. For now, though, it is harder to see them independently building a foundation model on that scale.
My judgment is therefore that the most important future breakthroughs in robotics will probably come from large-model companies, rather than robotics startups themselves.
Part II · The VGGT Series
6. Introducing the VGGT series
VGGT essentially follows the fully data-driven, black-box paradigm we just discussed. It takes several images, converts them into tokens, and aggregates information through global attention and frame-wise attention to model spatial relationships between views.

In VGGT, we attach multiple output branches to the aggregated tokens. These predict camera parameters, including pose information such as rotation and translation; depth; tracks; and other geometry-related representations.
Our central idea was to learn a geometry-aware representation. We thought the simplest way to do that was to keep the overall architecture simple while supplying sufficiently rich supervision at the outputs.
If a model is trained only to predict cameras, it may take a shortcut and learn features that solve the task without really encoding geometry. Predicting only depth can lead to similar issues. But if it must predict cameras, depth, tracks, and other geometric information together, the representation constrained by these tasks is more likely to be genuinely geometry-aware, rather than a shortcut solution.
VGGT is therefore fundamentally an exploration of how to learn this kind of geometry-aware representation.
Another important advantage is speed. Compared with traditional reconstruction methods, VGGT can be 100 or even 200 times faster in such comparisons, because it no longer depends on extremely time-consuming test-time optimization. This is also the problem VGGT aims to address: beyond reconstruction itself, the more important goal is to learn generalizable geometric representations.
For VGGT-Ω, we considered the many directions we could pursue next and asked which questions deserved priority.
We ultimately chose two main focuses.
The first was to study scaling.
Would reconstruction capabilities improve consistently as we increased training data, model size, and computing resources? We found that they would, which was a very encouraging signal.
The second focus was how to make scaling practical.
Scaling is not simply a matter of adding data and compute. As models grow, training efficiency and GPU memory consumption become problems; without addressing them, training may not even be feasible. We developed a series of targeted architectural designs and training strategies around these issues.
VGGT-Ω therefore primarily answers two questions: reconstruction models can scale, and making that scaling happen is not easy—it requires deliberate model design.
There is also a finding I particularly like: the tokens and features learned by these models really can help downstream tasks, including robotics.
We conducted a VLA experiment in the paper. It was relatively simple, since I am not a robotics expert, and we used an open-source OpenVLA implementation directly. [Editor's clarification: the paper's experiment uses the OpenVLA-OFT variant.]
Specifically, we added spatially aware tokens extracted from images by VGGT-Ω to OpenVLA. VGGT-Ω remained frozen throughout, with no parameter updates. In other words, it served as a geometric encoder.
We found that simply adding these spatial features produced a clear performance gain, including improved results on the LIBERO benchmark.
We had hoped to test the results on real robot platforms as well, but eventually realized that this was beyond my own expertise. Researchers specializing in robotics may need to explore that next.
7. Training data for VGGT
To summarize, I think the central point is that, provided data quality is maintained, more data produces more substantial improvements.
There is a crucial prerequisite, though: the data must not contain errors. We learned that lesson the hard way many times.
In VGGT-Ω, for example, we show some failure cases. Some datasets are postprocessed after collection using methods such as depth fusion. That processing can introduce errors, incorrectly merging an object's depth into another region. The paper includes a chair example: the seat should be above the floor, but after fusion its depth is incorrectly assigned the floor's depth.
This corrupted data gives the model very bad priors. In our internal analyses, many reproducible model errors could ultimately be traced back to incorrect training annotations. The model learns those erroneous patterns and transfers them to test data, even out-of-distribution data.
Data quality really is important. We all used to say that, but only after going through the complete cycle of training, analysis, and iteration did I realize that its importance far exceeded my expectations.
My second observation is that we still have too little 3D data.
Models can memorize particular data distributions. Sometimes a model performs less well in a special setting: it does not fail completely, but its results are clearly weaker than elsewhere. When we examined those cases closely, we often found a small amount of visually similar training data whose quality was not very good.
I used to have a rather idealized assumption: with enough high-quality data, the rules the model learns would naturally generalize and override the influence of poor-quality data. At least in today's reconstruction models, we have not clearly observed that happening.
My feeling is that the underlying reason is insufficient data. Today's models can still separate different data sources into independent regions of the distribution and memorize their patterns separately, instead of integrating them smoothly.
The two main lessons, then, are that data quality is extremely important, and that the scale of 3D data remains insufficient.
8. What counts as high-quality data?
An interesting property of 3D reconstruction is that we can often judge data quality directly through manual inspection.
Start with bad data. We typically back-project each view's depth using its camera parameters into 3D space. Nonflat walls, multiple ghosted layers of the same object, or positions that should coincide across views but do not are all typical signs of trouble.
What makes good data? An intuitive standard is whether it matches our understanding of the real three-dimensional world.
For a static scene, suppose a table corner reconstructed from 100 images aligns perfectly in 3D, with no obvious deviation. That suggests almost no error in the process of going from 2D to 3D and projecting back to 2D. Such results are usually beautiful.
The same applies to dynamic scenes. If a person walks forward at a constant speed, a good reconstruction gives a smooth, continuous 3D trajectory with consistent spatial relationships to the surroundings. An unreasonable discontinuity—suddenly jumping a long distance forward for no reason, for example—clearly does not fit the real world.
In one sentence, high-quality reconstruction data is data whose reconstructed result, when placed back in 3D space, agrees as closely as possible with our human intuitions about the real world.
9. Synthetic data
(VGGT uses a relatively high proportion of synthetic training data. Is synthetic data good or bad for reconstruction models?)
The synthetic data mainly comes from public datasets, and can be divided into several categories.
One category is rendered using graphics engines such as Blender or Unreal Engine (UE). We construct a scene containing various 3D assets, sample different camera trajectories within it, and render the corresponding RGB images, depth, and camera parameters.
Another category comes from games. Many projects, for example, obtain training data from games such as GTA. Because a game already contains a complete three-dimensional world, it can supply large amounts of data with precise geometry.
There is a subtle difference here from large language models when we ask whether synthetic data harms model quality.
In the LLM field, synthetic data often means data generated by a model itself or distilled from another model. It still comes from a model's distribution, so distribution shifts or accumulated errors can be concerns.
That is not what we mean by synthetic data in 3D reconstruction.
If the 3D assets in a scene are accurate, the scene is a small, completely and precisely defined 3D world. Synthetic data essentially consists of observations and samples of that world.
At the current stage, synthetic data in reconstruction is often high-quality data, rather than poor-quality data. Instead of degrading a model, it frequently helps it learn more accurate geometric regularities and improves performance on other data as well.
I have long wanted to test whether a sufficiently diverse and large synthetic dataset could produce a very accurate, beautiful reconstruction model. I think it might even outperform many existing models.
So, at least at this stage, the somewhat counterintuitive conclusion is that synthetic data is not the problem. It is an extremely important and valuable source of data for 3D reconstruction.
10. Is 3D data from wearable devices usable?
(You mentioned that real-world 3D data is scarce. Will the spread of wearable devices and robot sensors increase its availability?)
This is actually the kind of corrupted data I mentioned earlier. Its quality is often quite poor.
Consider depth collected by LiDAR-equipped iPhones. Although they can directly acquire depth, in our view their precision is insufficient for high-quality reconstruction. It is the sort of data we would want to filter out.
Many inexpensive RGB-D cameras have similar issues. Some RGB-D devices commonly used in industrial settings are, in our assessment, quite noisy and insufficiently accurate.
Even some indoor LiDAR systems do not deliver ideal data quality. High-end outdoor LiDAR can be genuinely accurate, but it is expensive and produces sparse data.
For high-precision reconstruction, much of the depth data people assume to be accurate is less reliable than they might imagine.
11. Other questions about VGGT's data
Holes are not really a problem. If a sensor can explicitly indicate uncertain regions and simply leave them empty, I find that acceptable.
Incorrect depth values are the real problem: the sensor appears to have measured a point, but the measured depth is wrong. This directly misleads the model and is far worse than a hole.
The training input is quite simple: a sequence of images, or an image stream. The outputs primarily comprise two things: camera parameters for each image, and depth for each image.
Camera parameters describe the camera's position and orientation in 3D space. Suppose you walk around a room holding a camera. Its position and viewing direction continually change. Turning left, turning right, or moving forward is reflected in those parameters.
They describe where the camera is and from which angle it observes the 3D world. With them, we can identify the viewpoint corresponding to each image and recover the scene's three-dimensional structure.
12. Data for VGGT-Ω
We substantially increased the data scale for Ω. The final training dataset comprised roughly four million sequences. These could come from videos or unordered image collections, just as with VGGT.
One of the most important lessons from Ω was the data cleaning I mentioned.
Before publishing the paper, we ran many internal tests. Overall performance was very good, but the model would suddenly break down in particular settings and produce very strange results. I was both frustrated and puzzled, because I did not know where the problem lay.
My collaborators and I spent a long time investigating. We checked the training code, checkpoints, and inference code—almost every conceivable source of error—but still could not find the cause.
Only when we reexamined the training data did we discover that the problem really was there. Sometimes a tiny amount of corrupted data, perhaps only 0.01% or even less, was enough for the model to learn an erroneous pattern that surfaced in particular settings.
Some issues remain unresolved. For example, when a person wears entirely black clothing in certain scenes, the model's prediction confidence for that person falls noticeably, and the results become more prone to errors.
We eventually linked this to the training data. To protect privacy, some datasets mask people, watermarks, logos, and similar regions in solid black. The model learns that pattern during training and is consequently affected by it.
We have not fully solved this problem, and discuss it explicitly in the paper.
13. Is data the biggest bottleneck in 3D research?
My sense is that progress in different directions is mutually reinforcing. But reconstruction is currently constrained to a considerable extent by data problems.
Looking at the history of LLMs, although many frontier labs now build their own data systems, there was a stage when people could train and evaluate fairly confidently using public datasets. Those datasets had been tested by the community over a long period, and repeatedly used and validated by many projects.
We still lack that kind of data infrastructure in 3D. There is not yet a large-scale, high-quality dataset that the whole community broadly trusts and has repeatedly validated.
I think robotics faces a similar problem. In recent conversations with robotics startups in the Bay Area, people frequently brought this up.
Different methods often use different training and test datasets. We also lack a unified evaluation standard for fair comparisons on real robot platforms.
From that perspective, robotics has similar data problems, and in some ways they may be even more serious than in 3D reconstruction.
Part III · Robotics
14. How can robotics researchers use VGGT?
If your task depends heavily on spatial structure, spatial relationships, or motion information, you can use our output tokens directly as conditioning, or add them as an extra input to your model.
That is probably the main point I want to share.
To put it more directly, robotics and vision researchers have long debated whether 3D understanding is necessary at all.
One camp thinks robots ultimately will not need explicit 3D representations, which may be replaced by more end-to-end methods. The other believes that progress in robot capabilities necessarily depends on explicit 3D representations.
What we are really discussing is spatial awareness. There are different routes to it: one relies on multiple sensors, such as RGB-D cameras and LiDAR; another is purely visual, learning to understand the 3D world from images alone.
Looking at Tesla's approach, many people naturally prefer the second route because it is easier to scale, costs less, and better suits large-scale deployment.
So the original question—whether 3D capabilities help other tasks—does not yet have a settled answer. The field is still debating it.
Personally, I believe they are valuable. At least for tasks that require understanding spatial structure, spatial relationships, and object motion, my view is clear: 3D capabilities are essential.
Both humans and robots ultimately live in a three-dimensional world, and we want robots to help us perform tasks within that world. But I do not believe explicit forms such as point clouds or depth maps are necessarily required. I have more faith in implicit 3D understanding: giving a model spatial awareness through tokens and features, without necessarily outputting traditional 3D representations directly.
My understanding as the author: obtaining 3D information does not necessarily require 3D sensors such as LiDAR or RGB-D cameras. It can come from multiple monocular viewpoints or a moving monocular camera. As long as we obtain the multiview information needed to recover 3D structure, a reconstruction model such as VGGT can recover it. Nor must this information exist explicitly as a rendered point cloud, mesh, or other 3D representation. The model is more like an encoder that encodes spatial information into tokens or features, giving it an understanding of spatial relationships and geometric structure.
15. Why does robotics rely so heavily on 2D rather than 3D?
(Much of today's robot training data relies on RGB images without depth, or on 2D video streams. “RGB in, action out” appears to be a mainstream approach.)
First, I would like to clarify a definition. Do you see a conflict between relying on 2D input and what we discussed earlier—introducing VGGT or another reconstruction model as an additional geometric encoder?
I can understand why people choose this route.
The first reason returns to our earlier point. Most tasks receiving attention today are relatively simple manipulation tasks, and may not genuinely require strong 3D understanding. With folding clothes, for example, I think sufficient training could enable good performance even without complete 3D information. Researchers are not deliberately avoiding 3D; at this stage, these tasks simply do not demand it as strongly.
The second reason is cost. RGB is the cheapest and easiest form of data to obtain. Adding any modality substantially increases collection and processing costs. From an engineering perspective, the RGB-only route is therefore naturally attractive.
The third reason is a personal observation: much of robotics today builds on developments in vision. Whether we are talking about VLAs or world models, many core capabilities were first demonstrated successfully in vision and then transferred to robotics. The robotics community therefore naturally tends to use mature 2D vision models directly, rather than introducing an additional 3D module.
I think there are good reasons to choose “RGB in, action out.” But that does not make 3D capabilities unimportant. The combination of today's task requirements, data costs, and technical trajectory is more likely what drives this choice.
16. Future research directions
I would consider expanding into robotics, but not robotics in its present sense. I believe robotics—or, more precisely, physical intelligence—will be the next major opportunity. Systems such as GPT, Gemini, and Codex are essentially pursuing digital intelligence. I strongly believe that intelligence will develop from the digital toward the physical world.
What I am less certain about is whether we should currently devote so much effort to specific robot tasks, such as all the subproblems within manipulation.
What really interests me is the scenario we discussed earlier: could a future model like the hypothetical “GPT-9.8,” when given any robot, rapidly understand it, learn to control it, and interact with the real world? I believe that day will come.
Rather than concentrating on one specific robot task, I am more interested in preparing for that day, moving the field in that direction, and ultimately making that scenario a reality.
Reconstruction itself is certainly fun and elegant, but I have worked on it for a long time. More than continuing to dig deeper into reconstruction alone, I want to consider how it can become part of the path toward the next generation of intelligent systems.
References
Editor's Note
After writing this blog for so long, I have still not formally introduced myself. Taking this interview with Jianyuan as an opportunity, here is a little about me: my name is Jianming Wang. At the time of writing, I am an Executive Director at Inno Angel Fund, focused primarily on robotics. Robot-related projects whose investments I have led or been deeply involved in include DEEP Robotics, MegaRobo, Sudo, Pokebot, Lumos Robotics, Noitom Robotics, and ULS Robotics. I began researching robotics investments in 2017, when the sector was still relatively overlooked. As of this year, that is approaching a decade.
Recently, I have been particularly interested in people with a deep understanding of 3D vision. There must be a reason that almost all animals on Earth have two eyes. [Editor’s clarification: this is the author’s intuitive observation, not a biological rule; the number and arrangement of eyes vary across animal species.]
We invest at a very early stage. Competition for investments is so intense today that several financing rounds can pass before you know it. My purpose in writing these articles is to find potential investments, whether in embodied AI, robotics, or AI more broadly.