Why a separate column on robot data
Large language models are trained on data from the internet. Robots have no ready-made source like that: every demonstration that carries action labels has to be produced by people, with equipment, in a physical space. The 2026 recipes of the leading models make the structure of the problem clear. More than 90% of pretraining hours come from the cheap layers (human video, handheld capture and simulation), and real-robot data makes up only 1–10% of them, yet in post-training the weighting swings back to real-robot data. This column asks who produces this data, how they produce it, what an hour costs, and how models mix it.
The six-layer data pyramid
The layers run from farthest to closest to the deployment distribution, and from lowest to highest unit cost. The common three-way split into "synthetic / UMI / teleoperation" is broken out here into six layers, for two reasons. Within "synthetic", physics engines and world models are two routes with entirely different failure modes. And by 2026, human video and deployment feedback had both become parts of leading model recipes that can no longer be ignored.
| Layer | Data | How it is produced | Typical scale | Role in the model |
|---|---|---|---|---|
| 1 | Human video and motion capture | Head-mounted cameras, wearables, mocap | On the order of 100,000 hours, with a free open-source base already available (Egocentric-10K / 100K / 1M) | Largest source for pretraining; "pseudo-actions" are recovered with SLAM plus hand-pose estimation |
| 2 | Handheld / UMI | Handheld gripper plus GoPro, no robot needed | 10,000–100,000 hours (Xiaomi-Robotics-1 used >100,000 hours) | The main cheap layer for pretraining; capture runs about 3× faster than teleoperation |
| 3 | Physics-simulation synthesis | Engines such as Isaac and MuJoCo, plus assets and GPUs | 780,000 trajectories in a single run (MimicGen / DexMimicGen) | An amplifier; locomotion relies on it almost 100%, manipulation is polarised |
| 4 | World-model generation | A video model generates "dreams", then actions are inferred from them | 240,000 clips for about 80,000 GPU hours (DreamGen / DreamDojo) | Three uses: generating data, serving as an RL environment, serving as an evaluator; the evidence is strongest for evaluation |
| 5 | Real-robot teleoperation | Real robot, operator, low-latency system and a site | Hundreds to a few thousand hours (Figure Helix about 500 hours; AgiBot World 2,976 hours) | The alignment layer: a small share of hours, a large share of training weight |
| 6 | Deployment feedback and real-robot RL | Autonomous rollouts, human takeovers and outcome labels | Almost no public datasets; the third AgiBot World release in 2026 is the first | Top of the pyramid: zero distance from the deployment distribution, but locked to both the embodiment and the policy |
Chapter 10, on each model company's data recipe and supplier landscape, breaks down 36 Chinese and US model companies layer by layer: which layers they use, how many hours, whether they collect in-house or buy, and from whom. It includes a company-by-layer heat map and a supplier matrix.
Three cross-cutting topics complete the column:
- Open datasets and formats: OXE, DROID, AgiBot World and LeRobot, plus the two generations of formats, RLDS and LeRobotDataset.
- Vendors, training grounds and policy: more than 70 training grounds in China, about RMB 4.47 billion raised by independent data vendors in one year, and the 2026 funding wave for US data-collection operators.
- Prices and recipes: per-source hours and sampling weights for each model, and price bands by region.
Hours by source and price per hour
Training data per model: share of hours by source
Cheap layers (human video, UMI, simulation) normally account for more than 90% of hours. Helix and LingBot take a pure real-robot route. The figure at the end of each bar is total hours.
| Model | Date | Human video h | UMI h | Sim h | World model h | Real robot h | Basis |
|---|---|---|---|---|---|---|---|
| Generalist GEN-1 | Apr 2026 | 500,000 | — | — | — | — | 500,000+ h of wearable data (company figure). Post-training uses about 1 h of robot data per task; the total is undisclosed and treated as 0 in the chart. |
| Shengshu Motus2 | Sep 2026 | 130,000 | — | — | — | 100 | 130,000 h of raw egocentric video; 100+ h of real-robot plus glove alignment data; simulation share not given. |
| Xiaomi-Robotics-1 | Jul 2026 | — | 101,000 | — | — | 7,200 | 100,000+ h of pretraining UMI plus 1,000+ h of instruction UMI for post-training; 7,200+ h of in-house real-robot data; real robot is 85% of post-training sampling weight. |
| NVIDIA EgoScale | Feb 2026 | 21,733 | — | — | — | 4 | 20,854 h egocentric + EgoDex 829 h + 50 h glove alignment data; 4 h on the robot. |
| NVIDIA GR00T N1 | Mar 2025 | — | — | 6,500 | 827 | 88 | The 6,500 h of simulation is the equivalent human-demonstration time of 780,000 trajectories (generated in about 11 h). Also includes OXE / AgiBot-Alpha and human video, with no hours given. |
| 1X 1XWM | 2025–26 | 900 | — | — | — | 70 | Mid-trained on 900 h of egocentric human video; 70 h of robot data for embodiment fine-tuning. |
| Ant Group LingBot-VLA | Jan 2026 | — | — | — | — | 20,000 | About 20,000 h of real-robot data across 9 dual-arm configurations. |
| Figure Helix | Feb 2025 | — | — | — | — | 500 | About 500 h of teleoperation (company figure). |
Data price bands: USD per hour
RMB converted at 1 USD = 7.1 CNY. The same "hour" may be raw or usable, and vendors differ on whether hardware amortisation and QA are included. Almost all figures come from vendors or the media.
| Type · region | Low | High | Basis |
|---|---|---|---|
| Egocentric · India, raw | $1 | $10 | Figure from the COO of Human Archive |
| Egocentric · vendor-collected (global) | $15 | $40 | Vendor figure |
| Egocentric · India, packaged and labelled | $15 | $50 | Industry-executive figure; down about 30% in six months |
| Egocentric · China | $42 | $42 | 36Kr estimate of RMB 300/h |
| Simulation synthesis · China | $28 | $70 | Media report of RMB 200–500/h |
| UMI without a robot · China | $42 | $92 | Huxiu RMB 300–400/h; 36Kr RMB 400–650/h |
| Real-robot teleop · global vendors | $28 | $60 | DexSet figure |
| Real-robot teleop · China | $70 | $141 | Media report of RMB 500–1,000/h |
| Real-robot teleop · specified target embodiment | $50 | $200 | DreamVu estimate (secondary source) |
Key findings
- The cheap layers are being pushed down to floor price by free open source. Build AI has released on the order of 100,000 hours of egocentric video under Apache 2.0. Of the 130,000 hours of egocentric data behind Motus2, about 87% comes from open source and 13% was purchased. The only premium left for commercial data vendors is stereo capture, labelling, gloves and task alignment.
- "Teleoperation costs an order of magnitude more than UMI" holds only for the equipment. Mobile ALOHA costs about $32k, against a bill of materials of about $73 for a UMI gripper. But collection rates differ by only about 3× (111 versus 35 demonstrations per hour), and there is no first-hand evidence on the price per demonstration. Chinese media figures: real-robot teleoperation RMB 500–1,000/hour, UMI RMB 300–400/hour, simulation RMB 200–500/hour.
- Recipes are rarely disclosed. Of more than 20 models, only Xiaomi-Robotics-1 and π0 give both hours by source and sampling weights. Share of hours and training sampling weight are different things: Xiaomi's post-training real-robot data is about 7% of total pretraining plus post-training hours, but 85% of post-training sampling weight.
- The simulators are all open source; the value goes to NVIDIA and to asset and service providers. Lightwheel (Guanglun Intelligence) is the largest commercial example, with about RMB 3.1 billion raised in a year, roughly 70% of all funding for China's independent data vendors. But its orders and revenue are company figures, and no sim-to-real correlation has been published.
- The evidence is strongest for world models as evaluators (SC3-Eval correlates with real-robot results at r≈0.93). As RL environments they add 10–25 percentage points, and the reward is still judged by the model itself. A market for "selling generated data" has not yet formed at any scale.
- Deployment feedback is the most expensive and least reusable layer. π*0.6 / RECAP is the only production-grade example with official first-hand numbers. The outsourcing layer (Enact, YC S26, and Cortex AI) has only just reached seed stage, and almost no public detail can be found on customer data-sovereignty terms.
- China's training-ground build-out is hot, while the business model remains unproven. By the end of Jun 2026 more than 70 sites were in operation and 46 under construction. At the Shougang Park data training centre, the original operator pulled out after less than 18 months. The Hubei centre's data deal with AgiBot was the first data transaction between companies.
Chapters
Every chapter follows the same skeleton: how the data is produced, vendors and channels, how models use it, scale, cost and price, limitations and disputes, and conflicting or unverified figures. On the original site, links in the text point to entity cards (companies, datasets, tools and formats, concepts, papers), and the cards link to one another.
| # | Chapter | What it covers |
|---|---|---|
| 01 | Real-robot teleoperation data | The alignment layer produced by real robots plus operators: hardware lineages, data-collection factories, effective yield and prices. |
| 02 | Handheld and UMI data | Capture with handheld grippers or wearables, no robot present: low cost and high throughput; by 2026 the main cheap layer for VLA pretraining. |
| 03 | Human video and motion-capture data | Egocentric video is dominated by free open source; mocap serves humanoid whole-body control; price bands are being pushed down. |
| 04 | Physics-simulation synthetic data | The cheap-layer amplifier that generates trajectories in bulk with physics engines: compute is nearly free, the cost lies in assets and QA. |
| 05 | World-model generated data | Video world models as data generators, RL environments and evaluators, and how strong the evidence is for each. |
| 06 | Deployment feedback and real-robot RL data | Real-robot rollouts, human takeovers and success/failure labels: who produces them, who buys them, how much is public. |
| 07 | Open datasets and data formats | A catalogue of open robot datasets, format toolchains, licences and recipe shares. |
| 08 | Data vendors, training grounds and policy | A matrix of Chinese and US embodied-data vendors, China's training grounds and policy, funding events and conflicting figures. |
| 09 | Data prices, costs and model recipes | Per-source hours and sampling weights by model, prices by type and region, equipment costs and scaling evidence. |
| 10 | Model companies' data recipes and supplier landscape | How about 40 Chinese and US embodied-model companies mix the six layers, whom they buy from, and who supplies whom. |
The column also includes 188 entity cards, browsable by type:
Sourcing and corrections
- Every number carries its source and access date. Self-reported company numbers are marked as company figures; numbers with only secondary sources, or no first-hand source found, are flagged as unverified.
- Existing claims this column has explicitly corrected:
- DROID's first-hand figure is 76,000 demonstrations, not 350,000+.
- The "about 3,800 hours" often quoted for OXE does not appear on any first-hand page.
- The Galaxea and Yizhuang plan is for 1 million hours (not 100 million), and 10 million hours over three years.
- No source can be found for 1X's "10,000+ hours of Redwood operational data".
- GR00T N1's 6,500 hours are equivalent human-demonstration time, not simulation run time.
- This section is personal research. Judgements hold as of the date written and are not investment or R&D advice.
The companion columns cover AI for Science data, LLM data and video-generation data; all four are indexed at AI Data.
Adapted from the Spark Science research notes (originally published in Chinese at jimmyresearch.com). Last updated Oct 14, 2026.