AI for Bio · Benchmark 01

CASP

Biennial blind test of protein structure prediction; remaining gains lie in complexes, RNA, ligands and ensembles.

bench-01Type AEvidence L4Structure & co-folding● Active No ranking中文

Overview

NameCASP · aliases: CASP, CASP14, CASP15, CASP16, CASP17
Full nameCritical Assessment of protein Structure Prediction
Ambiguous names (not auto-linked)None
FamilyNot a family entry. CASP1 to CASP17 are rounds of one challenge, not a family entry.
DomainStructure & co-folding (also: Small molecules, Genomics & RNA)
InclusionOriginal 30
TypeA · Biennial blind prediction challenge
Evidence levelL4 · Truth unpublished at submission
PublisherProtein Structure Prediction Center, University of California, Davis (Academic lab)
First release
Basis: First CASP round (CASP1), 1994; year-only evidence. The meeting month is not sourced and a conference end date is never used as first release. source
Other milestones
  • Challenge opens: · First CASP round (CASP1), 1994; year-only evidence. The meeting month is not sourced and a conference end date is never used as first release. source
Latest update · new round
Basis: CASP17 prediction season ended (official page)
CASP17 prediction season ended 2026-09-11; assessment under way; results due online 2026-11-30, meeting in Rome 2026-12-01 to 05.
Note: Dates differ across sources: official page says results online 11-30, meeting 12-01 to 12-05 source
The most recent sourced event, not a claim that nothing changed since.
Next expected† partly verified · results
Basis: Organizer announcement: CASP17 results go online
Note: CASP18 presumably 2028, not announced source
Status● Active · Active; CASP17 results pending; funding depends on few sponsors
ScaleCASP16: 210 registered groups (162 expert + 48 server); pharma ligand track 233 pose and 140 affinity targets.
  • Registered groups: 210 groups (CASP16)
  • Expert / server groups: 162 / 48 groups (CASP16)
  • Protein tertiary targets: 156 targets (CASP16)
  • Multimer targets: 108 targets (CASP16)
  • RNA targets: 52 targets (CASP16)
  • Pharma ligand pose targets: 233 targets (CASP16)
  • Pharma affinity targets (stage 2): 140 (110) targets (CASP16)
Wet-lab refreshFresh experiments each round · Biennial; truth comes from unpublished experimental structures donated by structural biologists; new truth only by entering the next round.
Own wet lab needed to evaluate a new modelNo, run it as a download
Models reported15 counted (board 3, retrospective 12, unconfirmed 1)
Unverified fields† next_expected, models_reported[0], models_reported[6], models_reported[15]
As of2026-11-16

What it measures

  • Task: Blind prediction, before release, of protein, complex, RNA, ligand pose and affinity, and ensemble structures from sequence.
  • Truth: X-ray, cryo-EM and NMR structures held unpublished until assessment.
  • Primary metric: Official per-category z-score ranking
  • Secondary metrics: Monomers: GDT_TS, lDDT; Complexes: QS, DockQ, ICS; RNA: TM-score, INF; Ligands: LDDT-PLI and affinity ranking
  • Version traps: Tracks change by round and assessors rotate, so rounds are not strictly comparable; models trained after a round are not blind to it.

Current ranking

No ranking

MetricCASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
Eval setCASP16 (2024); CASP17 results pending
Snapshot stale
Boardhttps://predictioncenter.org/casp17/index.cgi
Retrieved2026-11-16

Caveat. CASP17 results due 2026-11-30, no ranking yet. CASP16 has no single overall board; other site models mostly report retrospective numbers on CASP15/16 targets, not comparable.

No headline entry: CASP has no single overall board and CASP17 results are not out yet.

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
13AF3-serverrank 13 of 110 (protein domains, all); avg GDT_TS 87.1 (top-ranked models)CASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
CASP16 (2024); CASP17 results pending
Third-party
source CASP16 AF3 assessment paper (Proteins 2026), Table 2; AF2-based methods about 85.5
10AF3-serverrank 10 of 81 (protein multimers, all); avg DockQ 0.53 (first-ranked models)CASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
CASP16 (2024); CASP17 results pending
Third-party
source Same paper, Table 2; ranks 2nd on medium-difficulty oligomers
8AF3-serverrank 8 of 32 (RNA/DNA multimers)CASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
CASP16 (2024); CASP17 results pending
Third-party
source Same paper, Table 2; RNA monomer rank not cleanly extractable
40MassiveFoldrank 40 of 110 (domains); rank 30 of 81 (multimers)CASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
CASP16 (2024); CASP17 results pending
Third-party
source Same paper, Table 2
1trRosettaRNA-family server (academic group)No scoreCASP16 rank by official z-score (Z>-2) per category; GDT_TS (domains), DockQ (oligomers), TM-score (RNA)
CASP16 (2024); CASP17 results pending
Third-party
source Named top group for domains (GDT_TS); a trRosettaRNA-family server per site note, a group not a model; RNA and oligomer tops are other academic groups

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 15. A pairing whose evidence rests only on this survey's own page is shown as unverified.

Official leaderboard or results (3)

  • MassiveFold Board CASP16, five models per target selected by confidence · Group took part in CASP16 †unverified · evidence: no source link
  • OpenComplex2 Board · CASP16 participant with a talk at the CASP16 meeting; official ranks not found †partly verified · evidence: link
  • RhoFold+/DRfold2/trRosettaRNA Board CASP16 automated-server section (trRosettaRNA2); CASP15 retrospective (RhoFold+) · Family entry: trRosettaRNA2 (server) took part in CASP16; RhoFold+ is a CASP15 retrospective evaluation †partly verified · evidence: link

Retrospective dataset use (12)

  • AlphaFold 3 Third-party CASP15 public RNA targets · AF3 paper used CASP15 RNA targets post hoc; in CASP16 AF3 is only a component of other groups, no own entry · evidence: link
  • AQAffinity Third-party · CASP16 L1000/L3000 post hoc, figure only, L3000 overlaps training †partly verified · evidence: link
  • Boltz-2 Third-party affinity track, retrospective (2 targets / 140 pairs) · Retrospective CASP16 affinity data (2 targets, 140 pairs); not a blind participation · evidence: link
  • Chai-1 Third-party protein monomers of one past season · CASP15 monomer LDDT, second-hand quote only; post hoc †unverified · evidence: no source link
  • ESM3 Third-party CASP14 (71 targets) and CASP15 (70 targets) time-held-out structure and contact prediction · CASP14/15 time-split post hoc · evidence: link
  • IsoDDE Third-party · CASP16 blind affinity data used post hoc, figure only †partly verified · evidence: link
  • Nesso-1 Third-party · Added as a missing candidate in round 7 and confirmed in review. Resolution: Post-hoc use of the released affinity set (140 complexes, 2 targets), not participation in the blind round; the same rule applies to DPLM-2 × CAMEO. · evidence: link
  • NeuralPLexer3 Third-party · Matrix marks a self-report, but page evidence is only CASP15 RNA, duplicated from the RNA-Puzzles column †partly verified · evidence: link
  • OpenFold3 Third-party CASP16 protein and RNA monomers (post hoc) · CASP16 protein/RNA monomers, README figure only, post hoc †partly verified · evidence: link
  • Profluent-E1 Third-party · CASP15 contact maps, numbers not retrieved †partly verified · evidence: link
  • TerraBind Third-party · Retrospective CASP16 affinity evaluation · evidence: link
  • xTrimoPGLM Third-party · Post hoc on CASP15 (n=56) · evidence: link

Unconfirmed (1, not counted)

  • Enchant v1/v2 Unconfirmed source · v3 page claims held-out structures from this round, figure only, no protocol; split from a combined candidate and kept unconfirmed. low confidence †disputed · evidence: link

Caveats

  1. Complex, RNA and ligand tracks have few targets, so rankings hinge on a handful of targets and rarely carry confidence intervals.
  2. Single-chain accuracy is near experimental uncertainty; remaining headroom is in complexes, RNA, ligands and ensembles.
  3. Models trained after a round are not blind to its targets; post hoc use must check training cutoff against target release dates.
  4. Funding is fragile: after the NIH grant lapsed in 2025-07, a one-off donation from a leading developer in the field filled the gap (press report; amount undisclosed).
  5. Does not measure wet-lab hit rates; causal evidence from structure accuracy to discovery hit rates is weak.

How to use it

CASP17's prediction season is closed; the next round is presumably 2028 (not announced). Released CASP14–16 targets can serve as a retrospective set if the training cutoff predates target release; score with the official LGA, lDDT and OpenStructure tools. Report round, track, number of targets, model 1 versus best-of-5, and training cutoff.

Pairs well with: CAMEO · CAPRI · PoseBusters · RNA-Puzzles · ProteinGym

Sources

  1. CASP17 official page (official site)
  2. CASP16 official numbers page (official site)
  3. CASP1–9 round list (official site)
  4. CASP16 overview (Proteins 2026) (journal)
  5. CASP16 pharma ligand assessment (Proteins) (journal)
  6. AlphaFold 3 in CASP16 (Proteins 2026) (third-party paper)
  7. CASP14 domain assessment (Proteins 2021) (journal)
  8. STAT: CASP funding report (press)
  9. CASP17 meeting registration page (official site)

Corrections

4 corrections
FieldWasNowBasisFound
first_release1994-121994source2026-11-16
first_release_basisfirst CASP (CASP1) meeting at Asilomar, December 1994 (prediction season earlier in 1994)first CASP round, 1994 (year-only evidence; prediction season 1994). Meeting month not sourced; conference end date never used as first release.source2026-11-16
scale209 groups registered, ~300 targets, 200+ pharma co-crystals, ~160 affinity data210 groups registered (162 expert + 48 server); 233 pharma pose ligand targets; 140 pharma affinity targets (110 at stage 2); 156 tertiary + 108 multimer + 52 RNA targets; no single ~300-target figuresource2026-11-16
models_reported.model-nesso-1not listed (zero-pairing list)retrospective-dataset-usesource2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.