Overview
| Name | CAMEO · aliases: CAMEO, CAMEO3D, cameo3d.org |
|---|---|
| Full name | Continuous Automated Model EvaluatiOn |
| Ambiguous names (not auto-linked) | CAMEO 2022 |
| Family | Not a family entry. Single rolling evaluation service; static CAMEO subsets taken by papers are not family members. |
| Domain | Structure & co-folding |
| Inclusion | Original 30 |
| Type | A · Weekly automated rolling blind test, no human review |
| Evidence level | L4 |
| Publisher | SIB Swiss Institute of Bioinformatics, University of Basel, Biozentrum (Academic lab) |
| First release | † partly verified Basis: Maintainer papers state continuous weekly benchmarking since 2012; month of first targets not found Note: Year sourced; archive week count implies early 2012 only by arithmetic source |
| Other milestones |
|
| Latest update | No sourced date† · no later event Basis: No sourced event: cameo3d.org has no news or changelog No later sourced event found (as of 2026-11-16): cameo3d.org has no news or changelog and the weekly archive gives no dated release. Note: 2026-10-17 is only a Wayback snapshot date, not an update source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | Runs weekly on a rolling basis; no separately announced next event. |
| Status | ● Active · Active but reduced; single-chain 3D track ended April 2025 (unverified) |
| Scale | Jan 2024–Jun 2025: 12,205 targets over 76 weeks; about 160 targets per week.
|
| Wet-lab refresh | Fresh experiments each round · Weekly; truth is newly released PDB structures, about 160 per week, not produced for the benchmark. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 4 counted (board 1, retrospective 3, unconfirmed 1) |
| Unverified fields | † first_release, release_milestones.first_public_evidence, latest_update, status_note, ranking.entries[1], models_reported[0], models_reported[2] |
| As of | 2026-11-16 |
What it measures
- Task: Weekly blind evaluation of structure-prediction servers on PDB pre-release sequences; now mainly complex and ligand tracks.
- Truth: PDB structures released afterwards (X-ray, NMR, EM; resolution ≤4.0 Å).
- Primary metric: lDDT family (iLDDT, oligo-lDDT)
- Secondary metrics: QS-score; DockQ; lDDT-PLI (ligands)
- Version traps: Papers use different windows and filters, so state the window; a static 'CAMEO 2022' subset is not the live service.
Current ranking
Third-party
| Metric | lDDT / ligand success rate (CAMEO complex and ligand tracks) |
|---|---|
| Eval set | Mixed windows: CAMEO 2024-01-06 to 2025-06-14 (AF3 baseline run by CAMEO); CAMEO 2022-04 to 2022-06 static set (xTrimoPGLM) |
| Snapshot | stale |
| Board | https://cameo3d.org/ |
| Retrieved | 2026-11-16 |
Caveat. The live site renders server results dynamically, with no retrievable text board; single-chain 3D closed April 2025 (site note); entries use different windows and are not comparable.
No headline entry: No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the caveat text is shown instead.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | AlphaFold 3 (CAMEO baseline) | drug-like ligand success rate ~40%; antibody median LDDT 0.81 | lDDT / ligand success rate (CAMEO complex and ligand tracks) CAMEO 2024-01-06 to 2025-06-14 (76 weeks), AF3 v3.0.1 run by CAMEO as baseline | Third-party | source CAMEO assessment paper (Proteins 2025); best baseline in most ligand categories; behind stoichiometry-aware servers on oligomers; window end date inconsistent in paper. |
| — | xTrimoPGLM-Fold | TM-score 0.86 / GDT_TS 0.83 (ESMFold 0.85/0.81; AF2 0.87/0.85) | TM-score / GDT_TS CAMEO 2022-04 to 2022-06 static set (n=194) | Self-reported | source second-hand Developer report on a static subset; from site note, not re-verified. |
| — | Profluent-E1 | No score | Long-range contact precision CAMEO targets (window not stated in README) | Self-reported | source Shown only as a figure; no numeric score. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 4. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Official leaderboard or results (1)
- AlphaFold 3 Board operator-run complex and ligand baseline, 2024-01 to 2025-06 · CAMEO operators ran AF3 as a baseline (2025 assessment paper); maintainer-run, not a developer submission. †partly verified · evidence: link
Retrospective dataset use (3)
- DPLM-2 Third-party frozen 2022 target set, folding and inverse folding · Uses a frozen 2022 target set; no live submission, no service ranking. Resolution: The paper uses a frozen 2022 target set derived from the service, with no live submission and no service-assigned ranking. Recorded as a relation rather than deleted, so the rule matches Nesso-1 x CASP16. · evidence: link
- Profluent-E1 Third-party · CAMEO targets used post hoc; curve only. †partly verified · evidence: link
- xTrimoPGLM Third-party · CAMEO 2022-04 to 06 static set, n=194. · evidence: link
Unconfirmed (1, not counted)
- Enchant v1/v2 Unconfirmed source · Same source; static 2022 set, figure only. Unconfirmed; would count as retrospective use, not participation. low confidence †disputed · evidence: link
Caveats
- Windows differ between papers, so numbers are not comparable; difficulty tiers drift as servers improve, no confidence intervals are given, and new tools are hard to enter as servers.
- Pre-release data carry no stoichiometry, so homo-oligomers and monomers cannot be told apart, which affects complex scoring.
- Tools such as AlphaFold 3 have usage limits that hinder automated submission (second-hand report), so CAMEO runs it itself as a baseline.
- Truth is structure, not function or affinity; no direct evidence links lDDT gains to drug-discovery hit rates.
How to use it
No wet lab needed. Register as a server (receive targets automatically, return models on time), or replay contemporaneous new PDB structures offline and score with OpenStructure; the training cutoff must precede the window, and the window and filters must be stated. Not a basis for a single SOTA claim; pair with CASP, PoseBusters and PLINDER.
Pairs well with: CASP · PoseBusters · PLINDER / PINDER / Runs N' Poses · FoldBench
Sources
- CAMEO official site (official site)
- CAMEO assessment paper, Proteins 2025 (beyond single chains) (journal)
- CAMEO maintainer paper (Proteins) (journal)
- CAMEO 2018 paper (PubMed) (journal)
- xTrimoPGLM paper (developer) (preprint)
- cameo3d.org archive snapshot (2026-10-17) (archive)
Corrections
5 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| first_release | 2012 | 2012 | source | 2026-11-16 |
| latest_update | 2026-10-17 | null | source | 2026-11-16 |
| latest_update_desc | Wayback snapshot of cameo3d.org shows weekly benchmarking live; no dated news/changes retrievable, last substantive change undatable. | No dated update found: cameo3d.org carries no news or changelog and the weekly target archive gives no dated release. Must display as unverified. | source | 2026-11-16 |
| latest_update_kind | code-maintenance-only | none-since-release | source | 2026-11-16 |
| models_reported (DPLM-2) | No explained relation on the site | retrospective-dataset-use | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.