AI Data · Robotics

Robot and Embodied AI Data

A six-layer pyramid from human video to deployment feedback: who produces robot data, who sells it, how models mix it, and what an hour costs.

Spark Science ResearchUpdated Oct 14, 20269 min read
6
data layers, from human video to deployment feedback
>90%
of pretraining hours come from the cheap layers
1–10%
real-robot share of pretraining hours
188
entity cards across 10 chapters

Why a separate column on robot data

Large language models are trained on data from the internet. Robots have no ready-made source like that: every demonstration that carries action labels has to be produced by people, with equipment, in a physical space. The 2026 recipes of the leading models make the structure of the problem clear. More than 90% of pretraining hours come from the cheap layers (human video, handheld capture and simulation), and real-robot data makes up only 1–10% of them, yet in post-training the weighting swings back to real-robot data. This column asks who produces this data, how they produce it, what an hour costs, and how models mix it.

The six-layer data pyramid

The layers run from farthest to closest to the deployment distribution, and from lowest to highest unit cost. The common three-way split into "synthetic / UMI / teleoperation" is broken out here into six layers, for two reasons. Within "synthetic", physics engines and world models are two routes with entirely different failure modes. And by 2026, human video and deployment feedback had both become parts of leading model recipes that can no longer be ignored.

LayerDataHow it is producedTypical scaleRole in the model
1Human video and motion captureHead-mounted cameras, wearables, mocapOn the order of 100,000 hours, with a free open-source base already available (Egocentric-10K / 100K / 1M)Largest source for pretraining; "pseudo-actions" are recovered with SLAM plus hand-pose estimation
2Handheld / UMIHandheld gripper plus GoPro, no robot needed10,000–100,000 hours (Xiaomi-Robotics-1 used >100,000 hours)The main cheap layer for pretraining; capture runs about 3× faster than teleoperation
3Physics-simulation synthesisEngines such as Isaac and MuJoCo, plus assets and GPUs780,000 trajectories in a single run (MimicGen / DexMimicGen)An amplifier; locomotion relies on it almost 100%, manipulation is polarised
4World-model generationA video model generates "dreams", then actions are inferred from them240,000 clips for about 80,000 GPU hours (DreamGen / DreamDojo)Three uses: generating data, serving as an RL environment, serving as an evaluator; the evidence is strongest for evaluation
5Real-robot teleoperationReal robot, operator, low-latency system and a siteHundreds to a few thousand hours (Figure Helix about 500 hours; AgiBot World 2,976 hours)The alignment layer: a small share of hours, a large share of training weight
6Deployment feedback and real-robot RLAutonomous rollouts, human takeovers and outcome labelsAlmost no public datasets; the third AgiBot World release in 2026 is the firstTop of the pyramid: zero distance from the deployment distribution, but locked to both the embodiment and the policy

Chapter 10, on each model company's data recipe and supplier landscape, breaks down 36 Chinese and US model companies layer by layer: which layers they use, how many hours, whether they collect in-house or buy, and from whom. It includes a company-by-layer heat map and a supplier matrix.

Three cross-cutting topics complete the column:

  • Open datasets and formats: OXE, DROID, AgiBot World and LeRobot, plus the two generations of formats, RLDS and LeRobotDataset.
  • Vendors, training grounds and policy: more than 70 training grounds in China, about RMB 4.47 billion raised by independent data vendors in one year, and the 2026 funding wave for US data-collection operators.
  • Prices and recipes: per-source hours and sampling weights for each model, and price bands by region.

Hours by source and price per hour

Training data per model: share of hours by source

Cheap layers (human video, UMI, simulation) normally account for more than 90% of hours. Helix and LingBot take a pure real-robot route. The figure at the end of each bar is total hours.

Generalist GEN-1 Generalist GEN-1 · Human video / wearables / mocap: 100.0% 100% 500,000 h Shengshu Motus2 Shengshu Motus2 · Human video / wearables / mocap: 99.9% 100% Shengshu Motus2 · Real robot (teleop + alignment): 0.1% 130,100 h Xiaomi-Robotics-1 Xiaomi-Robotics-1 · Handheld UMI: 93.3% 93% Xiaomi-Robotics-1 · Real robot (teleop + alignment): 6.7% 7% 108,200 h NVIDIA EgoScale NVIDIA EgoScale · Human video / wearables / mocap: 100.0% 100% NVIDIA EgoScale · Real robot (teleop + alignment): 0.0% 21,737 h NVIDIA GR00T N1 NVIDIA GR00T N1 · Physics sim (equiv. hours): 87.7% 88% NVIDIA GR00T N1 · World-model generated: 11.2% 11% NVIDIA GR00T N1 · Real robot (teleop + alignment): 1.2% 7,415 h 1X 1XWM 1X 1XWM · Human video / wearables / mocap: 92.8% 93% 1X 1XWM · Real robot (teleop + alignment): 7.2% 7% 970 h Ant LingBot-VLA Ant LingBot-VLA · Real robot (teleop + alignment): 100.0% 100% 20,000 h Figure Helix Figure Helix · Real robot (teleop + alignment): 100.0% 100% 500 h
Human video / wearables / mocapHandheld UMIPhysics sim (equivalent hours)World-model generatedReal robot (teleop + alignment)
Hours are counted on different bases (equivalent hours, raw hours, company figures), so read only the order of magnitude of each share.
ModelDateHuman video hUMI hSim hWorld model hReal robot hBasis
Generalist GEN-1Apr 2026500,000————500,000+ h of wearable data (company figure). Post-training uses about 1 h of robot data per task; the total is undisclosed and treated as 0 in the chart.
Shengshu Motus2Sep 2026130,000———100130,000 h of raw egocentric video; 100+ h of real-robot plus glove alignment data; simulation share not given.
Xiaomi-Robotics-1Jul 2026—101,000——7,200100,000+ h of pretraining UMI plus 1,000+ h of instruction UMI for post-training; 7,200+ h of in-house real-robot data; real robot is 85% of post-training sampling weight.
NVIDIA EgoScaleFeb 202621,733———420,854 h egocentric + EgoDex 829 h + 50 h glove alignment data; 4 h on the robot.
NVIDIA GR00T N1Mar 2025——6,50082788The 6,500 h of simulation is the equivalent human-demonstration time of 780,000 trajectories (generated in about 11 h). Also includes OXE / AgiBot-Alpha and human video, with no hours given.
1X 1XWM2025–26900———70Mid-trained on 900 h of egocentric human video; 70 h of robot data for embodiment fine-tuning.
Ant Group LingBot-VLAJan 2026————20,000About 20,000 h of real-robot data across 9 dual-arm configurations.
Figure HelixFeb 2025————500About 500 h of teleoperation (company figure).

Data price bands: USD per hour

RMB converted at 1 USD = 7.1 CNY. The same "hour" may be raw or usable, and vendors differ on whether hardware amortisation and QA are included. Almost all figures come from vendors or the media.

$0 $50 $100 $150 $200 Egocentric · India, raw $1–10/h Egocentric · vendor-collected (global) $15–40/h Egocentric · India, packaged + labelled $15–50/h Egocentric · China ≈$42/h Simulation · China $28–70/h UMI, no robot · China $42–92/h Real-robot teleop · global vendors $28–60/h Real-robot teleop · China $70–141/h Real-robot teleop · specified embodiment $50–200/h
Egocentric videoSimulationUMIReal-robot teleop
Type · regionLowHighBasis
Egocentric · India, raw$1$10Figure from the COO of Human Archive
Egocentric · vendor-collected (global)$15$40Vendor figure
Egocentric · India, packaged and labelled$15$50Industry-executive figure; down about 30% in six months
Egocentric · China$42$4236Kr estimate of RMB 300/h
Simulation synthesis · China$28$70Media report of RMB 200–500/h
UMI without a robot · China$42$92Huxiu RMB 300–400/h; 36Kr RMB 400–650/h
Real-robot teleop · global vendors$28$60DexSet figure
Real-robot teleop · China$70$141Media report of RMB 500–1,000/h
Real-robot teleop · specified target embodiment$50$200DreamVu estimate (secondary source)

Key findings

  1. The cheap layers are being pushed down to floor price by free open source. Build AI has released on the order of 100,000 hours of egocentric video under Apache 2.0. Of the 130,000 hours of egocentric data behind Motus2, about 87% comes from open source and 13% was purchased. The only premium left for commercial data vendors is stereo capture, labelling, gloves and task alignment.
  2. "Teleoperation costs an order of magnitude more than UMI" holds only for the equipment. Mobile ALOHA costs about $32k, against a bill of materials of about $73 for a UMI gripper. But collection rates differ by only about 3× (111 versus 35 demonstrations per hour), and there is no first-hand evidence on the price per demonstration. Chinese media figures: real-robot teleoperation RMB 500–1,000/hour, UMI RMB 300–400/hour, simulation RMB 200–500/hour.
  3. Recipes are rarely disclosed. Of more than 20 models, only Xiaomi-Robotics-1 and π0 give both hours by source and sampling weights. Share of hours and training sampling weight are different things: Xiaomi's post-training real-robot data is about 7% of total pretraining plus post-training hours, but 85% of post-training sampling weight.
  4. The simulators are all open source; the value goes to NVIDIA and to asset and service providers. Lightwheel (Guanglun Intelligence) is the largest commercial example, with about RMB 3.1 billion raised in a year, roughly 70% of all funding for China's independent data vendors. But its orders and revenue are company figures, and no sim-to-real correlation has been published.
  5. The evidence is strongest for world models as evaluators (SC3-Eval correlates with real-robot results at r≈0.93). As RL environments they add 10–25 percentage points, and the reward is still judged by the model itself. A market for "selling generated data" has not yet formed at any scale.
  6. Deployment feedback is the most expensive and least reusable layer. π*0.6 / RECAP is the only production-grade example with official first-hand numbers. The outsourcing layer (Enact, YC S26, and Cortex AI) has only just reached seed stage, and almost no public detail can be found on customer data-sovereignty terms.
  7. China's training-ground build-out is hot, while the business model remains unproven. By the end of Jun 2026 more than 70 sites were in operation and 46 under construction. At the Shougang Park data training centre, the original operator pulled out after less than 18 months. The Hubei centre's data deal with AgiBot was the first data transaction between companies.

Chapters

Every chapter follows the same skeleton: how the data is produced, vendors and channels, how models use it, scale, cost and price, limitations and disputes, and conflicting or unverified figures. On the original site, links in the text point to entity cards (companies, datasets, tools and formats, concepts, papers), and the cards link to one another.

#ChapterWhat it covers
01Real-robot teleoperation dataThe alignment layer produced by real robots plus operators: hardware lineages, data-collection factories, effective yield and prices.
02Handheld and UMI dataCapture with handheld grippers or wearables, no robot present: low cost and high throughput; by 2026 the main cheap layer for VLA pretraining.
03Human video and motion-capture dataEgocentric video is dominated by free open source; mocap serves humanoid whole-body control; price bands are being pushed down.
04Physics-simulation synthetic dataThe cheap-layer amplifier that generates trajectories in bulk with physics engines: compute is nearly free, the cost lies in assets and QA.
05World-model generated dataVideo world models as data generators, RL environments and evaluators, and how strong the evidence is for each.
06Deployment feedback and real-robot RL dataReal-robot rollouts, human takeovers and success/failure labels: who produces them, who buys them, how much is public.
07Open datasets and data formatsA catalogue of open robot datasets, format toolchains, licences and recipe shares.
08Data vendors, training grounds and policyA matrix of Chinese and US embodied-data vendors, China's training grounds and policy, funding events and conflicting figures.
09Data prices, costs and model recipesPer-source hours and sampling weights by model, prices by type and region, equipment costs and scaling evidence.
10Model companies' data recipes and supplier landscapeHow about 40 Chinese and US embodied-model companies mix the six layers, whom they buy from, and who supplies whom.

The column also includes 188 entity cards, browsable by type:

60
companies and institutions
35
datasets
29
tools and formats
19
concepts and methods
45
papers and benchmarks

Sourcing and corrections

  • Every number carries its source and access date. Self-reported company numbers are marked as company figures; numbers with only secondary sources, or no first-hand source found, are flagged as unverified.
  • Existing claims this column has explicitly corrected:
    • DROID's first-hand figure is 76,000 demonstrations, not 350,000+.
    • The "about 3,800 hours" often quoted for OXE does not appear on any first-hand page.
    • The Galaxea and Yizhuang plan is for 1 million hours (not 100 million), and 10 million hours over three years.
    • No source can be found for 1X's "10,000+ hours of Redwood operational data".
    • GR00T N1's 6,500 hours are equivalent human-demonstration time, not simulation run time.
  • This section is personal research. Judgements hold as of the date written and are not investment or R&D advice.

The companion columns cover AI for Science data, LLM data and video-generation data; all four are indexed at AI Data.

Adapted from the Spark Science research notes (originally published in Chinese at jimmyresearch.com). Last updated Oct 14, 2026.