Overview
| Name | Virtual Cell Challenge and perturbation benchmarks · aliases: Virtual Cell Challenge, VCC, VCC 2025, VCC 2026, PerturBench, cell-eval |
|---|---|
| Full name | The Virtual Cell Challenge plus PerturBench and related single-cell perturbation benchmarks |
| Ambiguous names (not auto-linked) | Norman, Replogle, Adamson |
| Family | VCC is a yearly blind challenge while PerturBench and the Norman, Replogle and Adamson suites are static benchmarks and cell-eval is only a tool; the first release is the earliest member, PerturBench, not the VCC launch.
|
| Domain | Single-cell & perturbation (also: Genomics & RNA) |
| Inclusion | Original 30 |
| Type | A B C · A blind challenge plus static perturbation suites and an evaluation tool |
| Evidence level | L2–L4 · VCC uses a hidden test set; the static suites and baseline critiques are L2-L3 |
| Publisher | Arc Institute, Altos Labs (Nonprofit institute) |
| First release | · paper v1 Basis: Earliest family member: PerturBench's arXiv v1; VCC itself launched on 2025-06-26 source |
| Other milestones | |
| Latest update | † partly verified · new round Basis: Release of the VCC 2026 final test set, per the organizer's published schedule The VCC 2026 final test set was released and final submissions closed on 2026-11-05; as of 2026-11-16 no results have been published. Note: The date could not be matched to a dated official passage, so it is shown as unverified. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | † unverified · results Basis: The organizer schedule says the 2026 results come in mid-to-late November, while a press release says December source |
| Status | ● Active · Active: VCC 2026 has closed and its results are not yet published |
| Scale | VCC 2025: about 300,000 cells, 300 perturbations and 300+ teams in the final round; VCC 2026: six cell lines with cell counts unpublished.
|
| Wet-lab refresh | Fresh experiments each round · Yes, yearly: the organizers run their own Perturb-seq; the 2025 edition had about 300,000 cells and the 2026 cell count is unpublished. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 9 counted (board 1, modified protocol 6, retrospective 2) |
| Unverified fields | † latest_update, next_expected, scale.items[4], scale.items[5], ranking.entries[4], family.members[0] |
| As of | 2026-11-16 |
What it measures
- Task: Predict how a cell's transcriptome changes when a gene is knocked down, including unseen perturbations and unseen cell backgrounds, with the 2026 edition zero-shot.
- Truth: Newly measured CRISPRi Perturb-seq data from the organizers; the static suites use published Norman, Replogle and Adamson data.
- Primary metric: 2025: aggregate rank over PDS, DES and MAE
- Secondary metrics: Generalist Prize: mean rank across 7 metrics; 2026: a six-metric aggregate in the new cell-eval; Pearson delta and AUPRC
- Version traps: The 2025 data are now public and no longer blind, the two editions use different metric sets, and the L1/L2 forms of PDS are sensitive to prediction scale, so ranks do not compare across editions.
Current ranking
Board
| Metric | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics |
|---|---|
| Eval set | Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) |
| Snapshot | |
| Board | https://arcinstitute.org/news/virtual-cell-challenge-2025-wrap-up |
| Retrieved | 2026-11-16 |
Caveat. The 2026 edition closed on 2026-11-05 but had announced no standing as of 2026-11-16, so the current standing is still 2025; that board publishes ranks only, never scores.
Shown in the overview table: Board
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| 1 | Team BM_xTVC, xTrimoSCPerturb (improved scFoundation architecture with protein-model embeddings) | rank 1 of 300+ final-round teams (board publishes ranks only, no numeric scores) | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) | Board | source Overall winner; the organizer wrap-up publishes ranks and prizes only, with no numeric score for any team. |
| 2 | Team XLearning Lab (fully connected network, ESM-2 embeddings, residual delta prediction) | No score | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) | Board | source Not a site model; no numeric score published. |
| 3 | Team Outlier, TransPert (summary-level statistical framework) | No score | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) | Board | source Not a site model; no numeric score published. |
| — | Team Altos Labs, go-with-the-flow (flow matching with a custom U-Net), Generalist Prize | No score | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) | Board | source Not a site model; the prize goes by mean rank over 7 metrics among the top 50 final entries, with no numbers. |
| — | STATE | Withheld: A different protocol and not a board result, so nothing is shown in the ranking row. | VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) | Self-reported | source Not an entry: the organizers used four metrics from its paper to define the Generalist Prize, and its self-reported Tahoe-100M results use a different protocol and are not board results. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 9. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Official leaderboard or results (1)
- xTrimoSCPerturb / scFoundation Board · Overall winner of VCC 2025 per the organizer wrap-up; scores unpublished. · evidence: link
Developer report, modified protocol (6)
- C2S-Scale Self-reported · Uses its own datasets and metrics, not the VCC protocol. · evidence: link
- CellFM Self-reported · Adamson and Norman data through GEARS, not VCC. · evidence: link
- Stack Self-reported · Own perturbation sets; its VCC claim is rejected on the model page. · evidence: link
- STATE Self-reported · Self-reported with cell-eval on Tahoe-100M; its VCC blind score is unpublished. · evidence: link
- Tahoe-x1 Self-reported · An own benchmark, explicitly not VCC. · evidence: link
- X-Cell Self-reported · Own held-out iPSC set; VCC participation unconfirmed. · evidence: link
Retrospective dataset use (2)
- Lingshu-Cell Third-party · A post-hoc evaluation on the VCC H1 leaderboard rather than participation. · evidence: link
- scGPT / Geneformer Third-party · Independent third-party evaluations on Norman and Adamson data. · evidence: link
Caveats
- The 2025 metrics can be gamed by scale and transformations: a third-party analysis shows L1/L2 PDS is sensitive to prediction scale and almost all submissions scored worse than baseline on MAE.
- Several independent evaluations find deep and foundation models do not reliably beat mean, no-change or linear baselines on perturbation prediction.
- The board gives ranks without numbers, and with 300 perturbations the organizers never state whether rank gaps exceed noise.
- The 2025 edition used a single cell line so cross-cell generalization was untested, and the full data are now public and no longer blind.
- External pretraining and public Perturb-seq may overlap the test targets, and there is no prospective wet-lab validation nor evidence linking scores to target-discovery hit rates.
How to use it
The 2025 data are public and can be run statically, but a real blind test means entering the next edition. Always report mean, no-change and linear baselines, use several metrics such as cosine PDS, Pearson delta and AUPRC plus PerturBench ranks and RMSD, and never rely on a single PDS. Generalization to one's own cell lines needs new Perturb-seq, whose cost is undisclosed.
Pairs well with: DREAM5 / BEELINE / CausalBench · scIB (+Open Problems) · Basenji/Enformer test set
Sources
- Organizer wrap-up page for VCC 2025 (official site) ·
- VCC 2026 announcement (official site) ·
- PerturBench paper (preprint) ·
- Third-party comparison of deep models against linear baselines (third-party paper)
- Third-party analysis of the PDS metric (third-party paper)
- PertEval-scFM (paper)
Corrections
4 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| first_release | 2025 (VCC launch treated as the family's first release) | 2024-08-20, PerturBench arXiv v1, with the VCC 2025 launch (2025-06-26) kept as a member | source | 2026-11-16 |
| scale | 2026 edition: six cell lines, zero-shot, cell counts implied | 2025 figures verified; the 2026 cell count is unpublished and 'six cell lines' was not re-confirmed | source | 2026-11-16 |
| latest_update | VCC 2026 in progress, without a dated event | 2026-10-22 final test set released; submissions closed 2026-11-05; winners unannounced as of 2026-11-16 (flagged unverified) | source | 2026-11-16 |
| models_reported | STATE, Stack, X-Cell and others listed as VCC results | self-reported-modified-protocol: own datasets and metrics, not the VCC board | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.