AI Data · Video

Video Generation Data

Seven layers of training data for video generation models: open web video, owned platforms, licensed libraries, synthetic renders, curation and captioning, post-training preferences, and new modalities. Who supplies it, how models use it, what it costs, and the legal lines.

Spark Science ResearchUpdated Oct 15, 202611 min read
10
chapters
134
entity cards
7
data layers
31
models mapped

Why a separate column on video generation data

The algorithms behind video generation models converged quickly over 2024–2026: a diffusion Transformer, an in-house captioner and preference alignment form the common skeleton of almost every technical report. What separates the models is the data: where it comes from, whether it can be used legally, how heavily it is processed, and which categories were added for a particular direction.

Yet this is exactly what companies disclose least. Of 31 mainstream models, only a few open models plus Meta Movie Gen, NVIDIA Cosmos and Stability's Stable Video Diffusion (LVD) give a clear account of their data; almost all the closed frontier models say only "public and licensed data". This column pieces together the facts that can be verified: for each layer of data, who produces it, who sells it, how models use it, what it costs, and which legal line it runs into.

Seven layers and three cross-cutting themes

The layers are ordered by distance from the final model distribution and by production method. Layers 1–3 are where the raw material comes from, layers 4–6 are how it is made and processed, and layer 7 covers the specialised categories added in 2025–2026.

LayerDataMain sourcesTypical scaleRole in the model
1Public web video and open datasetsYouTube scraping; one lineage: HD-VILA-100M → Panda-70M / Koala-36M → OpenVid-1MOpen datasets at tens of millions of clips; closed sets "billions", undisclosedMain raw material for pretraining; open datasets today serve mostly as a research and evaluation base
2Owned platforms and UGCKuaishou, Douyin / TikTok, YouTube, Meta, Tencent, AlibabaPlatform libraries of tens of billions of videos (not a training-set figure)Never directly acknowledged in tech reports; the only company-confirmed case is Google using a YouTube subset to train Veo (2 / 3 / 3.1)
3Licensed libraries and marketplacesShutterstock, Getty Images, Adobe Stock, Troveo AI, Protege (incl. Calliope), Visual China Group (VCG), Jiecheng (Jetsen)Creator footage at $1–4 per minute; overall price band spans two orders of magnitudeCompliance supplement and high-quality source footage; Adobe Firefly Video and Moonvalley's Marey sell themselves as "fully licensed"
4Synthetic, rendered and distilled dataUE5 / game engines, Objaverse renders, VLM-generated captions, few-step distillationOnly Cosmos discloses rendered video, at 4%Used mainly on the text side (recaptioning) and for camera, motion and physics controllability labels
5Cleaning, splitting, scoring and captioningIn-house pipelines: shot detection → filtering → dedup and balancing → in-house captionerCosmos: 20M hours → about 100M clipsDetermines the "usable hours"; almost nobody discloses funnel retention
6Post-training: SFT, preferences and reward modelsHand-curated SFT, pairwise preferences, multi-dimensional reward modelsTencent HunyuanVideo: about 1M SFT examples; VideoReward / VideoAlign: 182K preference tripletsA small, expensive human step; the main entry point for outside annotation vendors
7New data categoriesJoint audio-video, multi-shot, reference pairs, camera / 3D, game actions, animeSpatialVID: 7,089 hours; Phantom-Data: about 1M pairsFour categories are constructed algorithmically from large pools; game-action data is the only new buy-and-sell market

Three cross-cutting chapters complete the column:

  • Data recipes and suppliers by model: a heatmap of 31 Chinese and US models × seven layers, and 29 supply relationships.
  • Copyright litigation and regulation: studios suing Midjourney and MiniMax, the YouTube creator lawsuits, the EU training-data summary, and China's Interim Measures and labelling rules.
  • Vendors and prices: licensing unit prices, AI licensing revenue at listed companies, and funding of data companies.

Which layers each model uses

Data layers used by each video model company

Darkest = the layer is the main pretraining source or a self-built collection system; light = minor use or open-source only; grey = not used or not disclosed. Hover over a cell for details; the full figures are in the table below.

Web /open videoOwnplatformLicensedlibrariesSynthetic /renderedPipeline /captionerSFT /preferenceNewmodalitiesCHINAKuaishou KlingKuaishou Kling · Web / open video: clearly usedKuaishou Kling · Own platform: not used / undisclosedKuaishou Kling · Licensed libraries: not used / undisclosedKuaishou Kling · Synthetic / rendered: minor or open-source onlyKuaishou Kling · Pipeline / captioner: main layerKuaishou Kling · SFT / preference: not used / undisclosedKuaishou Kling · New modalities: minor or open-source onlyByteDance Seedance / JimengByteDance Seedance / Jimeng · Web / open video: not used / undisclosedByteDance Seedance / Jimeng · Own platform: not used / undisclosedByteDance Seedance / Jimeng · Licensed libraries: minor or open-source onlyByteDance Seedance / Jimeng · Synthetic / rendered: not used / undisclosedByteDance Seedance / Jimeng · Pipeline / captioner: main layerByteDance Seedance / Jimeng · SFT / preference: clearly usedByteDance Seedance / Jimeng · New modalities: minor or open-source onlyByteDance Seaweed-7B / WaverByteDance Seaweed-7B / Waver · Web / open video: not used / undisclosedByteDance Seaweed-7B / Waver · Own platform: not used / undisclosedByteDance Seaweed-7B / Waver · Licensed libraries: not used / undisclosedByteDance Seaweed-7B / Waver · Synthetic / rendered: clearly usedByteDance Seaweed-7B / Waver · Pipeline / captioner: main layerByteDance Seaweed-7B / Waver · SFT / preference: main layerByteDance Seaweed-7B / Waver · New modalities: minor or open-source onlyShengShu ViduShengShu Vidu · Web / open video: minor or open-source onlyShengShu Vidu · Own platform: not used / undisclosedShengShu Vidu · Licensed libraries: clearly usedShengShu Vidu · Synthetic / rendered: minor or open-source onlyShengShu Vidu · Pipeline / captioner: not used / undisclosedShengShu Vidu · SFT / preference: not used / undisclosedShengShu Vidu · New modalities: clearly usedAIsphere PixVerseAIsphere PixVerse · Web / open video: not used / undisclosedAIsphere PixVerse · Own platform: minor or open-source onlyAIsphere PixVerse · Licensed libraries: not used / undisclosedAIsphere PixVerse · Synthetic / rendered: not used / undisclosedAIsphere PixVerse · Pipeline / captioner: minor or open-source onlyAIsphere PixVerse · SFT / preference: not used / undisclosedAIsphere PixVerse · New modalities: minor or open-source onlyMiniMax HailuoMiniMax Hailuo · Web / open video: not used / undisclosedMiniMax Hailuo · Own platform: not used / undisclosedMiniMax Hailuo · Licensed libraries: not used / undisclosedMiniMax Hailuo · Synthetic / rendered: not used / undisclosedMiniMax Hailuo · Pipeline / captioner: minor or open-source onlyMiniMax Hailuo · SFT / preference: not used / undisclosedMiniMax Hailuo · New modalities: not used / undisclosedAlibaba Tongyi WanxiangAlibaba Tongyi Wanxiang · Web / open video: clearly usedAlibaba Tongyi Wanxiang · Own platform: clearly usedAlibaba Tongyi Wanxiang · Licensed libraries: not used / undisclosedAlibaba Tongyi Wanxiang · Synthetic / rendered: clearly usedAlibaba Tongyi Wanxiang · Pipeline / captioner: main layerAlibaba Tongyi Wanxiang · SFT / preference: clearly usedAlibaba Tongyi Wanxiang · New modalities: minor or open-source onlyTencent HunyuanVideoTencent HunyuanVideo · Web / open video: clearly usedTencent HunyuanVideo · Own platform: not used / undisclosedTencent HunyuanVideo · Licensed libraries: not used / undisclosedTencent HunyuanVideo · Synthetic / rendered: minor or open-source onlyTencent HunyuanVideo · Pipeline / captioner: main layerTencent HunyuanVideo · SFT / preference: main layerTencent HunyuanVideo · New modalities: clearly usedStepFun Step-Video-T2VStepFun Step-Video-T2V · Web / open video: clearly usedStepFun Step-Video-T2V · Own platform: not used / undisclosedStepFun Step-Video-T2V · Licensed libraries: not used / undisclosedStepFun Step-Video-T2V · Synthetic / rendered: not used / undisclosedStepFun Step-Video-T2V · Pipeline / captioner: main layerStepFun Step-Video-T2V · SFT / preference: clearly usedStepFun Step-Video-T2V · New modalities: not used / undisclosedSand AI MAGI-1Sand AI MAGI-1 · Web / open video: clearly usedSand AI MAGI-1 · Own platform: not used / undisclosedSand AI MAGI-1 · Licensed libraries: not used / undisclosedSand AI MAGI-1 · Synthetic / rendered: not used / undisclosedSand AI MAGI-1 · Pipeline / captioner: main layerSand AI MAGI-1 · SFT / preference: not used / undisclosedSand AI MAGI-1 · New modalities: minor or open-source onlyZhipu CogVideoXZhipu CogVideoX · Web / open video: main layerZhipu CogVideoX · Own platform: not used / undisclosedZhipu CogVideoX · Licensed libraries: not used / undisclosedZhipu CogVideoX · Synthetic / rendered: not used / undisclosedZhipu CogVideoX · Pipeline / captioner: main layerZhipu CogVideoX · SFT / preference: not used / undisclosedZhipu CogVideoX · New modalities: not used / undisclosedKunlun Tech SkyReelsKunlun Tech SkyReels · Web / open video: clearly usedKunlun Tech SkyReels · Own platform: not used / undisclosedKunlun Tech SkyReels · Licensed libraries: main layerKunlun Tech SkyReels · Synthetic / rendered: minor or open-source onlyKunlun Tech SkyReels · Pipeline / captioner: main layerKunlun Tech SkyReels · SFT / preference: clearly usedKunlun Tech SkyReels · New modalities: minor or open-source onlyHPC-AI Tech Open-Sora 2.0HPC-AI Tech Open-Sora 2.0 · Web / open video: clearly usedHPC-AI Tech Open-Sora 2.0 · Own platform: not used / undisclosedHPC-AI Tech Open-Sora 2.0 · Licensed libraries: not used / undisclosedHPC-AI Tech Open-Sora 2.0 · Synthetic / rendered: not used / undisclosedHPC-AI Tech Open-Sora 2.0 · Pipeline / captioner: main layerHPC-AI Tech Open-Sora 2.0 · SFT / preference: not used / undisclosedHPC-AI Tech Open-Sora 2.0 · New modalities: not used / undisclosedBaidu MuseSteamerBaidu MuseSteamer · Web / open video: not used / undisclosedBaidu MuseSteamer · Own platform: not used / undisclosedBaidu MuseSteamer · Licensed libraries: not used / undisclosedBaidu MuseSteamer · Synthetic / rendered: not used / undisclosedBaidu MuseSteamer · Pipeline / captioner: minor or open-source onlyBaidu MuseSteamer · SFT / preference: minor or open-source onlyBaidu MuseSteamer · New modalities: minor or open-source onlyUNITED STATESOpenAI Sora / Sora 2OpenAI Sora / Sora 2 · Web / open video: minor or open-source onlyOpenAI Sora / Sora 2 · Own platform: not used / undisclosedOpenAI Sora / Sora 2 · Licensed libraries: clearly usedOpenAI Sora / Sora 2 · Synthetic / rendered: not used / undisclosedOpenAI Sora / Sora 2 · Pipeline / captioner: minor or open-source onlyOpenAI Sora / Sora 2 · SFT / preference: not used / undisclosedOpenAI Sora / Sora 2 · New modalities: minor or open-source onlyGoogle VeoGoogle Veo · Web / open video: not used / undisclosedGoogle Veo · Own platform: main layerGoogle Veo · Licensed libraries: minor or open-source onlyGoogle Veo · Synthetic / rendered: clearly usedGoogle Veo · Pipeline / captioner: main layerGoogle Veo · SFT / preference: not used / undisclosedGoogle Veo · New modalities: clearly usedGoogle Genie 3Google Genie 3 · Web / open video: not used / undisclosedGoogle Genie 3 · Own platform: not used / undisclosedGoogle Genie 3 · Licensed libraries: not used / undisclosedGoogle Genie 3 · Synthetic / rendered: not used / undisclosedGoogle Genie 3 · Pipeline / captioner: not used / undisclosedGoogle Genie 3 · SFT / preference: not used / undisclosedGoogle Genie 3 · New modalities: clearly usedMeta Movie GenMeta Movie Gen · Web / open video: not used / undisclosedMeta Movie Gen · Own platform: minor or open-source onlyMeta Movie Gen · Licensed libraries: minor or open-source onlyMeta Movie Gen · Synthetic / rendered: clearly usedMeta Movie Gen · Pipeline / captioner: main layerMeta Movie Gen · SFT / preference: main layerMeta Movie Gen · New modalities: main layerRunwayRunway · Web / open video: clearly usedRunway · Own platform: not used / undisclosedRunway · Licensed libraries: clearly usedRunway · Synthetic / rendered: not used / undisclosedRunway · Pipeline / captioner: minor or open-source onlyRunway · SFT / preference: not used / undisclosedRunway · New modalities: minor or open-source onlyLuma RayLuma Ray · Web / open video: not used / undisclosedLuma Ray · Own platform: not used / undisclosedLuma Ray · Licensed libraries: not used / undisclosedLuma Ray · Synthetic / rendered: not used / undisclosedLuma Ray · Pipeline / captioner: not used / undisclosedLuma Ray · SFT / preference: not used / undisclosedLuma Ray · New modalities: minor or open-source onlyPikaPika · Web / open video: not used / undisclosedPika · Own platform: not used / undisclosedPika · Licensed libraries: not used / undisclosedPika · Synthetic / rendered: not used / undisclosedPika · Pipeline / captioner: not used / undisclosedPika · SFT / preference: not used / undisclosedPika · New modalities: not used / undisclosedMidjourney VideoMidjourney Video · Web / open video: minor or open-source onlyMidjourney Video · Own platform: minor or open-source onlyMidjourney Video · Licensed libraries: minor or open-source onlyMidjourney Video · Synthetic / rendered: not used / undisclosedMidjourney Video · Pipeline / captioner: not used / undisclosedMidjourney Video · SFT / preference: not used / undisclosedMidjourney Video · New modalities: not used / undisclosedAdobe Firefly VideoAdobe Firefly Video · Web / open video: not used / undisclosedAdobe Firefly Video · Own platform: not used / undisclosedAdobe Firefly Video · Licensed libraries: main layerAdobe Firefly Video · Synthetic / rendered: not used / undisclosedAdobe Firefly Video · Pipeline / captioner: not used / undisclosedAdobe Firefly Video · SFT / preference: not used / undisclosedAdobe Firefly Video · New modalities: not used / undisclosedMoonvalley MareyMoonvalley Marey · Web / open video: not used / undisclosedMoonvalley Marey · Own platform: not used / undisclosedMoonvalley Marey · Licensed libraries: main layerMoonvalley Marey · Synthetic / rendered: not used / undisclosedMoonvalley Marey · Pipeline / captioner: not used / undisclosedMoonvalley Marey · SFT / preference: not used / undisclosedMoonvalley Marey · New modalities: not used / undisclosedNVIDIA CosmosNVIDIA Cosmos · Web / open video: clearly usedNVIDIA Cosmos · Own platform: not used / undisclosedNVIDIA Cosmos · Licensed libraries: clearly usedNVIDIA Cosmos · Synthetic / rendered: minor or open-source onlyNVIDIA Cosmos · Pipeline / captioner: main layerNVIDIA Cosmos · SFT / preference: clearly usedNVIDIA Cosmos · New modalities: main layerApple video modelApple video model · Web / open video: minor or open-source onlyApple video model · Own platform: not used / undisclosedApple video model · Licensed libraries: minor or open-source onlyApple video model · Synthetic / rendered: minor or open-source onlyApple video model · Pipeline / captioner: not used / undisclosedApple video model · SFT / preference: not used / undisclosedApple video model · New modalities: not used / undisclosedAmazon Nova ReelAmazon Nova Reel · Web / open video: minor or open-source onlyAmazon Nova Reel · Own platform: not used / undisclosedAmazon Nova Reel · Licensed libraries: minor or open-source onlyAmazon Nova Reel · Synthetic / rendered: not used / undisclosedAmazon Nova Reel · Pipeline / captioner: not used / undisclosedAmazon Nova Reel · SFT / preference: not used / undisclosedAmazon Nova Reel · New modalities: not used / undisclosedGLOBAL / EUROPELightricks LTXLightricks LTX · Web / open video: main layerLightricks LTX · Own platform: not used / undisclosedLightricks LTX · Licensed libraries: clearly usedLightricks LTX · Synthetic / rendered: not used / undisclosedLightricks LTX · Pipeline / captioner: main layerLightricks LTX · SFT / preference: minor or open-source onlyLightricks LTX · New modalities: minor or open-source onlyStability SVDStability SVD · Web / open video: main layerStability SVD · Own platform: not used / undisclosedStability SVD · Licensed libraries: not used / undisclosedStability SVD · Synthetic / rendered: minor or open-source onlyStability SVD · Pipeline / captioner: main layerStability SVD · SFT / preference: clearly usedStability SVD · New modalities: minor or open-source onlyBlack Forest Labs FLUX 3Black Forest Labs FLUX 3 · Web / open video: not used / undisclosedBlack Forest Labs FLUX 3 · Own platform: not used / undisclosedBlack Forest Labs FLUX 3 · Licensed libraries: not used / undisclosedBlack Forest Labs FLUX 3 · Synthetic / rendered: not used / undisclosedBlack Forest Labs FLUX 3 · Pipeline / captioner: not used / undisclosedBlack Forest Labs FLUX 3 · SFT / preference: not used / undisclosedBlack Forest Labs FLUX 3 · New modalities: clearly usedSynthesiaSynthesia · Web / open video: minor or open-source onlySynthesia · Own platform: not used / undisclosedSynthesia · Licensed libraries: main layerSynthesia · Synthetic / rendered: not used / undisclosedSynthesia · Pipeline / captioner: minor or open-source onlySynthesia · SFT / preference: not used / undisclosedSynthesia · New modalities: not used / undisclosed
Main layerClearly usedMinor or open-source onlyNot used / undisclosed
Based on each company's papers, technical reports and official blogs; company figures and secondhand sources are marked in the table.

Disclosed figures by model

CompanyModelRegionRecipe publicDisclosed figures and route
Kuaishou KlingKling 1.x–3 / OmniChinaPartialWeb/open: Koala-36M, 172K h (open dataset, paper figure). Pipeline: three-tier governance + VTSS, no totals. Only the Koala-36M and Omni reports outline the pipeline; no scale for the main model; UGC and licensing undisclosed.
ByteDance Seedance / JimengSeedance 1.0 / 2.0ChinaPartialLicensed: says “public and licensed libraries”, no detail (company figure). SFT: several hundred categories, three reward models, no counts. The 1.0 report details the pipeline but not its scale; the 2.0 page has no data information; Douyin UGC use is inferred.
ByteDance Seaweed-7B / WaverSeaweed-7B / WaverChinaPublicAbout 100M clips ≈ 220K h (our conversion); millions of simulated videos; SFT about 700K (50K weighted); Waver saw 200M+ clips. ByteDance’s two most number-rich reports: over 500K hours processed per day, invalid-clip rate cut from 42% to 2.9%.
ShengShu ViduVidu Q1–Q3 / VidarChinaNot publicVidar uses about 750K open-source robot episodes; licensed: a dedicated VCG zone (company figure). No training-data report; no platform of its own, so it relies on licensed libraries, open data and embodied-AI partners; most leads are local reports pending verification.
AIsphere PixVersePixVerse V5 / V6 / C1 / RChinaNot publicWhether its 150M user records are used for training is undisclosed (inferred). Mentions only an “in-house video understanding and labelling system”; the R-series real-time world model would need action-conditioned data (inferred).
MiniMax HailuoHailuo 01 / 02 / 03ChinaNot publicHailuo 02 trained on about 4× the data of its predecessor (company figure). No tech report; lawsuit allegations include copyrighted film and TV characters, which are not findings of fact.
Alibaba Tongyi WanxiangWan 2.1 / 2.2 / 2.5ChinaPublic“Billions of images and videos”, images about 10× videos; hundreds of millions of rendered text images; millions each of simple- and complex-motion videos; 2.2 adds +83.2% video (company figure). “Internal copyrighted sources + publicly accessible data”; four filtering steps remove about 50%; human scoring on 100 clusters.
Tencent HunyuanVideoHunyuanVideo 1 / 1.5ChinaPublicRaw >10M h → about 800M clips (1.5); v1 SFT about 1M; 1.5 continued training 1M per task; v1 video-to-audio 250K h pretraining / 80K h SFT. One of the best-disclosed Chinese open models on scale, though sources are not named (claims GDPR compliance, includes synthesized data).
StepFun Step-Video-T2VStep-Video-T2VChinaPublic2B video-text pairs; SFT candidate pool about 30M → human curation. Eight-frame quality sampling + optical flow + 120K-cluster balancing; a vertical model co-built with Zhongguang Tianze is not evidence of corpus use.
Sand AI MAGI-1MAGI-1ChinaPartial“Tens of PB” of raw video and images. Sources and final scale undisclosed; autoregressive per-second captions; post-training volume not found.
Zhipu CogVideoXCogVideoX / QingyingChinaPublicAbout 35M clips after filtering (≈58K h, our conversion). “Most of the videos available online” + LAION-5B / COYO; 20K labels used to train six classifiers.
Kunlun Tech SkyReelsSkyReels V1 / V2ChinaPublicSelf-collected 280K+ films and 800K+ TV episodes, about 6.2M h (licensing status undisclosed); about 1M SFT subset. The most direct copyright exposure in the table; SkyCaptioner-V1 human-checked accuracy 76.3%.
HPC-AI Tech Open-Sora 2.0Open-Sora 1.x / 2.0ChinaPublicThree stages with 70M / 10M / 5M samples. Total cost $199.6K (about 4,160 H200 GPU-days); video sources not named.
Baidu MuseSteamerMuseSteamerChinaNot public“Cleaning of hundreds of millions of Chinese multimodal data items” (media figure). Sources, scale and SFT undisclosed; integrated Chinese audio-video.
OpenAI Sora / Sora 2Sora / Sora 2USNot publicLicensed: six-year Shutterstock deal (company figure). “Public or licensed”; on Mar 24, 2026 OpenAI announced it would stop the Sora app (effective shutdown date unverified) and the Disney deal ended.
Google VeoVeo 2 / 3 / 3.1USPartialOwn platform: a subset of YouTube’s ~20B videos (CNBC, secondhand); joint audio-video training (model card). Model card mentions multi-level Gemini captions, dedup and synthetic captions; video count and hours undisclosed.
Google Genie 3Genie 3USNot publicReal-time interactive worlds, 720p / 24 fps (secondhand). Training data undisclosed.
Meta Movie GenMovie GenUSPublicO(100)M videos + O(1)B images (paper figure); audio pretraining O(1)M hours. The most complete data account among US closed models: curation + LLaMa3-Video captioner + human-refined SFT; the paper does not give source composition.
RunwayGen-3 / Gen-4 / AlephUSNot publicNearly 4,000 YouTube channels (404 Media leaked sheet; not necessarily all ingested); custom model on the Lionsgate library (20K+ titles, secondhand). Leak and studio partnership coexist; Getty / AMC licensing not found.
Luma RayRay2 / Ray3USNot publicTraining set, licensing and post-training all undisclosed.
PikaPikaUSNot publicNo training data, licensing or litigation found.
Midjourney VideoVideo V1USNot publicCourt filing says it needs “billions of public images”; user ratings (image side; video undisclosed). Technology licensing deal with Meta (Midjourney as licensor); sued by Disney and two other studios.
Adobe Firefly VideoFirefly VideoUSPartialAverage purchase price $2.62/min (Bloomberg, secondhand). Company line: licensed content, no web scraping, no customer content; video data composition undisclosed.
Moonvalley MareyMareyUSPartialLicensed hours undisclosed. Says it is fully licensed and signs creators directly; Troveo says it supplies most of the data (Troveo’s claim).
NVIDIA CosmosCosmos WFMUSPublic20M raw hours → about 100M pretraining clips; rendered synthetic 4%; about 10M clips for fine-tuning. Paper: “proprietary + public web video”; sued over alleged use of YouTube videos referenced by HD-VILA and others (no ruling).
Apple video modelApple (unnamed)USNot publicShutterstock licence (reported); sued over alleged use of Panda-70M; video details undisclosed.
Amazon Nova ReelNova ReelUSNot publicOnly a generic “licensed / proprietary / open source / public”; sued by YouTube creators in Apr 2026.
Lightricks LTXLTX-Video / LTX-2GlobalPartialShutterstock research licence (company figure). “Large amounts of public data supplemented by licensed material”; LTX-2 weights are open, the data is not.
Stability SVDStable Video DiffusionGlobalPublicLVD: 580M clips / 212 years of footage (paper figure); 250K high-quality clips for fine-tuning. LVD source described only as “a collection of long videos”; neither itemized nor released.
Black Forest Labs FLUX 3FLUX 3EuropeNot publicVideo prediction is 95%+ of training compute (company figure, secondhand). Joint image / video / audio / action; sources and scale undisclosed.
SynthesiaSynthesia avatarsGlobalPartialPaid actors filmed in studio + licensed media (company figure). Actors paid per clip, provenance documents retained; share of public content undisclosed.

Key findings

  1. The raw material is largely shared; the differences lie in processing and added categories. Most mainstream open video datasets trace back to the same pool of YouTube videos (the HD-VILA-100M lineage), and the pretraining bases of the four leading Chinese closed models are also very similar. The real differences are in captioner specs, filtering thresholds and the data added for a particular direction: Kling's film/TV and dialect voices, Seedance's multimodal references, Vidu's anime and embodied data, PixVerse's real-time interaction. The base is shared; each direction pays its own tax (see the chapter on data recipes by model).
  2. Disclosure runs backwards: the bigger the player, the less it says. Tech reports uniformly say "public + licensed / internal copyrighted sources", and no primary document admits to using the company's own platform UGC. Data sources have been exposed mainly by media investigations (the leaked Runway training-data sheet, the Cosmos scraping controversy, The Atlantic's investigation of video datasets) and by discovery in lawsuits.
  3. The licensing market is growing, but it has not become the main channel for training data. Shutterstock's data business reached about $200M in 2025 and then started declining year on year; at Visual China Group, value-added services including AI training data are only 5.9% of revenue; Chinese stock-library companies generally do not report AI corpus revenue separately. Licensing serves more as a compliance supplement, a litigation settlement tool and a source of high-bitrate footage than as the bulk of pretraining.
  4. Synthetic data is used for labels, not pixels. Almost every model uses a VLM to generate synthetic captions. The value of rendered video lies in controllability labels such as camera pose, motion and physics; training purely on synthetic video hurts dynamics (Hunyuan-GameCraft ablation). Training on competitors' outputs is prohibited by terms of service, but no public enforcement case was found.
  5. Among the new categories, only game-action data has formed a seller's market. Data that aligns frames with controller inputs does not exist on the web, and in 2026 specialised suppliers such as General Intuition, Worldmodeldata and Origin Lab appeared. Audio-video, multi-shot and reference-pair data are all constructed algorithmically from large pools.
  6. The legal lines are still unsettled, but traceable provenance has become a cost. As of Oct 2026 there has not been a single merits ruling in video generation. Rights holders have shifted from trying to stop training to trading for output-side governance and licences: the MPA–ByteDance MOU, and the Disney × OpenAI deal, which ended when OpenAI announced in Mar 2026 that it would stop Sora. The EU training-data summary, China's "lawful source" principle and the YouTube creator lawsuits (DMCA §1201) together raise the risk of data without a chain of title.

How to read the column

  • To find out what data a particular model used, go straight to the heatmap and recipe table in Data recipes and suppliers by model, then to the model card.
  • To find out who sells and at what price, see Licensed libraries and marketplaces and Vendors and prices.
  • Each chapter ends with a table of conflicting and unverified figures. Companies' own numbers are marked as company figures; anything not independently verified is flagged ⚠️. All litigation and deal statuses are as of Oct 2026.

Chapters

#ChapterWhat it covers
01Public web video and open datasetsLineage, licences and litigation risk of open video datasets: most sources converge on HD-VILA and YouTube.
02Owned platforms and UGC dataWho owns video corpora, what user terms say, and whether tech reports admit using platform data.
03Licensed libraries and marketplacesThe market for licensing video to AI trainers: price bands, listed-company revenue, contract forms and China–US differences.
04Synthetic, rendered and distilled dataSynthetic data is used mainly for text and supervision signals; rendered video is a tiny share and mostly undisclosed.
05Cleaning, splitting, scoring and captioning pipelinesThe standard pipeline from raw video to training clips, funnel retention and caption specs.
06Post-training: SFT, preference annotation and reward modelsHow fine-tuning sets, preference pairs, reward models and eval data are produced, and where annotation vendors really fit in.
07New data categories: audio-video, multi-shot, reference pairs and interactive world modelsSix new kinds of training data beyond text-video pairs: how they are made, how big they are, who sells them.
08Data recipes and suppliers by modelWhat 31 Chinese and US video models disclose about their data, and who supplies whom.
09Copyright litigation and regulationLawsuits over video generation training data, the reach of precedents and the rules in each jurisdiction, and their effect on procurement.
10Vendors and pricesCross-layer vendor comparison, listed-company licensing revenue, funding events and a normalised video data price table.

The column also maintains 134 entity cards: 36 companies and institutions, 21 datasets, 11 tools and formats, 24 concepts and methods, 8 papers and benchmarks, and 34 models. These deep pages have not been migrated yet.

Disclaimer

This column is a personal research compilation. Every figure is labelled in the text with its source and basis; companies' self-reported data is marked as a company figure, and anything not independently verified is flagged ⚠️. All judgements are as of the date of writing and are not investment or R&D advice.

Adapted from the Spark Science research notes (originally published in Chinese at jimmyresearch.com). Last updated Oct 15, 2026.