- Evidence level
- L4*
- First release
- Latest update
- new round
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 15
AI for Bio benchmarks: what 44 tests can and cannot tell you
A survey of 44 widely used AI-for-biology benchmarks and the 112 models released or updated in the last year that are measured on them: what each test measures, which models actually report it, whether new wet-lab data keeps flowing in, and where the training data comes from. Every benchmark and model has its own page. Scores appear only in each benchmark's ranking, verbatim and never across benchmarks.
The 44 benchmarks: identity, freshness and current ranking
One column per benchmark. The last row is each benchmark's current ranking; metrics and protocols differ by column, so scores cannot be compared across columns. As of 2026-11-16; each cell shows its own snapshot date.
Skip to the by-benchmark table| Domain | Structure & co-folding | Small molecules | Proteins | Antibodies | Genomics & RNA | Single-cell & perturbation | Molecule generation | Text, LLMs & agents | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Benchmark | 01 CASP | 05 PoseBusters | 15 CAMEO | 22 CAPRI | 25 RNA-Puzzles | 30 PLINDER / PINDER / Runs N' Poses family | 31 FoldBench | 03 MoleculeNet (+OGB) | 04 TDC (ADMET + DTI) family | 06 CASF-2016 | 07 USPTO family (+PaRoutes) family | 12 DUD-E / LIT-PCBA family | 19 PMO (Practical Molecular Optimization) | 20 CACHE | 27 Polaris / ASAP / OpenADMET | 33 PoseX | 44 FS-Mol | 02 ProteinGym family | 17 CAFA | 26 FLIP / FLIP2 family | 28 Adaptyv competitions / BenchBB family | 40 MotifBench | 34 AbBiBench | 35 FLAb / FLAb2 | 36 AIntibody | 43 Ginkgo GDPa1 | 08 Basenji/Enformer test set family | 11 ClinVar hold-out protocol | 13 CAGI | 16 GUE + Nucleotide Transformer benchmark family | 21 DREAM5 / BEELINE / CausalBench family | 37 TraitGym | 38 BEND | 39 RNAGym | 09 scIB (+Open Problems) family | 24 Virtual Cell Challenge and perturbation benchmarks family | 10 GuacaMol (+MOSES) family | 32 CrossDocked2020 family | 14 PubMedQA | 18 LAB-Bench / LABBench2 family | 23 BioASQ | 29 BixBench | 41 BioML-bench | 42 BioMysteryBench |
| Type | A | C B | A | A | A B | C B | B | B | B | B | B | B | D | A | A C | B | B | B | A | B | A B | D | B | B | A | A B | B | B | A | B | A B C | B | B | B | B C | A B C | D | D | B E | E | A B | E | E | E |
| Evidence level | L4* | L3* | L4 | L4* | L2–L4* | L3* | L3* | L2–L3* | L2–L3* | L2* | L2–L3* | L2* | L2* | L5* | L4* | L3* | L3* | L2–L3* | L4–L5* | L2–L3* | L5* | L2* | L2–L3* | L2–L3* | L5* | L4* | L2–L3* | L2–L3* | L4–L5* | L2–L3* | L2–L5* | L3* | L2* | L2* | L2* | L2–L4* | L2* | L2* | L1 | L1–L2* | L1–L4* | L2* | L2* | L2–L4* |
| First release | † | † | † | † | † | † | † | † | † | † | † | † | † | † | † | |||||||||||||||||||||||||||||
| Latest update | new round | maintenance only | No sourced date† no later event | new round | † new round | † new version | board entry | paper revision | paper revision | no later sourced event found (as of 2026-11-16) | data | † maintenance only | † new version | † data | † new version | paper revision | † archived | † data | † new round | † new version | new round | board entry | paper revision | maintenance only | data | † data | † maintenance only | data | new round | data | † paper revision | † maintenance only | new version | † board entry | new version | † new round | maintenance only | † data | maintenance only | maintenance only | results | † board entry | paper revision | † board entry |
| Status | Active | Active | Active | Round in progress | Low-frequency | Active | Low-frequency | Frozen | Active | Frozen | Frozen | Low-frequency | Low-frequency | Active | Low-frequency | Active | Frozen | Low-frequency | Active | Low-frequency | Active | Active | Active | Active | Low-frequency | Low-frequency | Active | Active | Active | Low-frequency | Low-frequency | Low-frequency | Active | Active | Active | Active | Frozen | Frozen | Frozen | Active | Active | Active | Low-frequency | Active |
| Wet-lab refresh | New | Static | New | New | New | Accumulating | Static | Static | Static | Static | Static | Static | Static | New | New | Static | Static | Accumulating | Accumulating | Static | New | Static | Static | Accumulating | New | New | Static | Accumulating | New | Static | Static | Static | Static | Static | Accumulating | New | Static | Static | Static | Static | Accumulating | Static | Static | Static |
| Models reported | 15 | 8 | 4 | 2 | 3 | 5 | 12 | 2 | 4 | 0 | 5 | 2 | 2 | 0 | 0 | 6 | 0 | 10 | 0 | 2 | 4 | 3 | 6 | 1 | 1 | 0 | 3 | 5 | 1 | 0 | 0 | 3 | 0 | 1 | 1 | 9 | 0 | 1 | 2 | 3 | 0 | 3 | 1 | 1 |
| Current ranking | No ranking CASP has no single overall board and CASP17 results are not out yet. | Not comparable 5 results on different protocols, no single ranking | Third-party No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the c… | No ranking There is no ranking this round, and the entries come from the CASP16 assessment of the joint round rather tha… | Board Kaggle private-board rank 1 team 0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10) Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles Board · Best site model: AlphaFold 3 | Board success_rate 15.068; rmsd_success_rate 17.808; mean_lddt_pli 0.304; coverage 96.053 success_rate: share of predictions with LDDT-PLI > 0.8 and BiSyRMSD < 2; the page also lists rmsd_success_rate, mean_lddt_pli and coverage · Runs N' Poses (Polaris preview benchmark, plinder-org/runs-n-poses) Board · | Board Protein-Protein 72.93% / Antibody-Antigen 47.90% / Protein-Ligand 64.90%; Protein-RNA 62.32%, Protein-DNA 79.18%, RNA Monomer 0.61, DNA Monomer 0.53 Interface success rate (% ; DockQ>=0.23, protein-ligand requires LRMSD<2 A and LDDT-PLI>0.8); monomers use LDDT · FoldBench official repo leaderboard, full set (targets released after 2023-01) Board · | Not comparable 3 results on different protocols, no single ranking | Board None of the 22 official boards dates its entries, so every score is withheld and no entry can serve as headli… | Not comparable 4 results on different protocols, no single ranking | Not comparable 5 results on different protocols, no single ranking | Third-party No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);… | Self-reported Sum 18.362 (Table 3, 'Goal-directed hit generation results', 23 PMO tasks, AUC top-10) AUC top-10, summed over 23 oracles (max 23), 10,000-call budget · PMO 23 oracles (QED, DRD2, GSK3B, JNK3 plus 19 GuacaMol-derived), oracle functions via TDC Self-reported · stale not independently reproduced | No ranking The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a… | No ranking Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET… | Third-party No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);… | Not comparable 1 result on different protocols, no single ranking | Board AIDO Protein-RAG (16B) 0.518 (SE 0.0) Average Spearman (zero-shot, DMS substitutions) · ProteinGym v1.3, 217 substitution DMS assays, zero-shot Board · stale Best site model: VenusREM (#2) | No ranking CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline… | Not comparable 3 results on different protocols, no single ranking | No ranking The current round has no results and the organiser page shows no ranks or scores. | Board RFD4-Proteina (indexed + medium thinking) 55.06 (± 1.35) MotifBench score (0-100) · 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline Board · Best site model: RFdiffusion3 (indexed) (#2) | Board ProteinMPNN 0.29 Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays Board · Best site model: Boltz-2 (ipSAE) (#4) | Third-party The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is sho… | Board AuraIDE / AuraBind (Aureka Biotechnologies, internal model) KD 94.7 pM (KinExA); best experimental antibody 113 pM Measured KD (HT-SPR and KinExA) plus five developability screens; placement per task · AIntibody round 1, 3 challenges, 511 designs from 29 organizations, SARS-CoV-2 spike RBD; all designs made and measured in one lab Board · | Board Anonymous 0.708 Spearman correlation per property on the private held-out set · AbDev competition 2025, 80 private held-out antibodies, 5 properties Board · | Self-reported The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given. | Not comparable 5 results on different protocols, no single ranking | Third-party ClinPred / MetaRNN AUROC 0.92 AUROC (pathogenic vs benign missense) · CAGI6 Annotate-All-Missense full evaluation set (60 predictors) Third-party · stale | No ranking No official board and no site model reports on these suites, so there is no headline number. | Not comparable 3 results on different protocols, no single ranking | Board CADD 0.712 (se 0.038) AUPRC weighted-averaged by chromosome (main metric), with standard error · TraitGym Mendelian traits (matched_9) and Complex traits (matched_9), subset 'all', zero-shot and linear-probing tracks Board · stale Best site model: Evo2 40B (#5) | Board SpliceAI 0.674 Per-task metrics with no single official ranking metric: conditional log loss (bits/base, lower better), macro Spearman correlation, top-1 tissue identification accuracy · BEND 2.0 Table 1 snapshot: 8 endpoints across 5 capability categories, 17 model-interface rows Board · Best site model: AlphaGenome (#2) | Board 0.2120 (Ribozyme 0.1081 / tRNA 0.4310 / Aptamer 0.0970) Signed Spearman correlation, macro-averaged over three ncRNA categories with equal category weight (from v0.1.1) · RNAGym fitness leaderboard, ncRNA only (31 assays); mRNA coding/splicing assays excluded Board · † | Board The board orders by scaled overall mean but shows no numeric overall score; derived means are not used as a h… | Board rank 1 of 300+ final-round teams (board publishes ranks only, no numeric scores) VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics · Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) Board · | No ranking No official board and no site model reports a GuacaMol score, so no headline is given. | Not comparable 3 results on different protocols, no single ranking | Board 82.0 Accuracy (%) (macro-F1 mostly not reported), reasoning-required setting · PQA-L test (500 questions) Board · stale | Board 0.609 (Tools + high); Base 0.345 LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' · LABBench2 (1,912 tasks, 11 categories) Board · | Board mckpt2 Yes/No F1 0.9481, Acc. 0.9524; Factoid MRR 0.2000; List F1 0.1537 Yes/no macro-F1 as the ranking key of Tables 7 and 8, plus factoid MRR and list F1 · BioASQ Task 14b batch 2 exact answers, phase A+ (Table 7) and phase B (Table 8); preliminary results Board · † Best site model: MedQA-1 (university team; phase B uses a frontier-LLM ensemble) (#3) | Board 0.523 Average score over the successor's 20 tasks, scored programmatically against source papers · BixBench3 (successor benchmark, 20 research-study-scale tasks) Board · | Self-reported The source has no rank-1 entry: entries are unranked or protocol-specific, so the caveat is shown as for a no… | Third-party 83.70% Final-answer accuracy, mean of three full runs, scored by a judge model · Third-party board run: Terminus 2 harness, 200 turns / 4 h per task Third-party · † |
Structure & co-folding · 7 benchmarks · board 3 · third-party 1 · not comparable 1 · no ranking 2
- Evidence level
- L3*
- First release
- Latest update
- maintenance only
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 8
- Evidence level
- L4
- First release
- †
- Latest update
- No sourced date†no later event
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 4
- Evidence level
- L4*
- First release
- †
- Latest update
- new round
- Status
- Round in progress
- Wet-lab refresh
- New
- Models reported
- 2
- Evidence level
- L2–L4*
- First release
- Latest update
- †new round
- Status
- Low-frequency
- Wet-lab refresh
- New
- Models reported
- 3
- Evidence level
- L3*
- First release
- †
- Latest update
- †new version
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 5
- Evidence level
- L3*
- First release
- Latest update
- board entry
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 12
Small molecules · 10 benchmarks · board 1 · self-reported 1 · third-party 2 · not comparable 4 · no ranking 2
- Evidence level
- L2–L3*
- First release
- Latest update
- paper revision
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 2
- Evidence level
- L2–L3*
- First release
- Latest update
- paper revision
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 4
- Evidence level
- L2*
- First release
- Latest update
- no later sourced event found (as of 2026-11-16)
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 0
- Evidence level
- L2–L3*
- First release
- Latest update
- data
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 5
- Evidence level
- L2*
- First release
- Latest update
- †maintenance only
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 2
- Evidence level
- L2*
- First release
- Latest update
- †new version
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 2
- Evidence level
- L5*
- First release
- †
- Latest update
- †data
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 0
- Evidence level
- L4*
- First release
- Latest update
- †new version
- Status
- Low-frequency
- Wet-lab refresh
- New
- Models reported
- 0
- Evidence level
- L3*
- First release
- Latest update
- paper revision
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 6
- Evidence level
- L3*
- First release
- Latest update
- †archived
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 0
Proteins · 5 benchmarks · board 2 · not comparable 1 · no ranking 2
- Evidence level
- L2–L3*
- First release
- †
- Latest update
- †data
- Status
- Low-frequency
- Wet-lab refresh
- Accumulating
- Models reported
- 10
- Evidence level
- L4–L5*
- First release
- †
- Latest update
- †new round
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 0
- Evidence level
- L2–L3*
- First release
- Latest update
- †new version
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 2
- Evidence level
- L5*
- First release
- †
- Latest update
- new round
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 4
- Evidence level
- L2*
- First release
- Latest update
- board entry
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 3
Antibodies · 4 benchmarks · board 3 · third-party 1
- Evidence level
- L2–L3*
- First release
- Latest update
- paper revision
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 6
- Evidence level
- L2–L3*
- First release
- Latest update
- maintenance only
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 1
- Evidence level
- L5*
- First release
- Latest update
- data
- Status
- Low-frequency
- Wet-lab refresh
- New
- Models reported
- 1
- Evidence level
- L4*
- First release
- †
- Latest update
- †data
- Status
- Low-frequency
- Wet-lab refresh
- New
- Models reported
- 0
Genomics & RNA · 8 benchmarks · board 3 · self-reported 1 · third-party 1 · not comparable 2 · no ranking 1
- Evidence level
- L2–L3*
- First release
- †
- Latest update
- †maintenance only
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 3
- Evidence level
- L2–L3*
- First release
- †
- Latest update
- data
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 5
- Evidence level
- L4–L5*
- First release
- †
- Latest update
- new round
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 1
- Evidence level
- L2–L3*
- First release
- Latest update
- data
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 0
- Evidence level
- L2–L5*
- First release
- †
- Latest update
- †paper revision
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 0
- Evidence level
- L3*
- First release
- Latest update
- †maintenance only
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 3
- Evidence level
- L2*
- First release
- Latest update
- new version
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 0
- Evidence level
- L2*
- First release
- Latest update
- †board entry
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 1
Single-cell & perturbation · 2 benchmarks · board 2
- Evidence level
- L2*
- First release
- Latest update
- new version
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 1
- Evidence level
- L2–L4*
- First release
- Latest update
- †new round
- Status
- Active
- Wet-lab refresh
- New
- Models reported
- 9
Molecule generation · 2 benchmarks · not comparable 1 · no ranking 1
- Evidence level
- L2*
- First release
- Latest update
- maintenance only
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 0
- Evidence level
- L2*
- First release
- †
- Latest update
- †data
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 1
Text, LLMs & agents · 6 benchmarks · board 4 · self-reported 1 · third-party 1
- Evidence level
- L1
- First release
- Latest update
- maintenance only
- Status
- Frozen
- Wet-lab refresh
- Static
- Models reported
- 2
- Evidence level
- L1–L2*
- First release
- Latest update
- maintenance only
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 3
- Evidence level
- L1–L4*
- First release
- †
- Latest update
- results
- Status
- Active
- Wet-lab refresh
- Accumulating
- Models reported
- 0
- Evidence level
- L2*
- First release
- †
- Latest update
- †board entry
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 3
- Evidence level
- L2*
- First release
- Latest update
- paper revision
- Status
- Low-frequency
- Wet-lab refresh
- Static
- Models reported
- 1
- Evidence level
- L2–L4*
- First release
- Latest update
- †board entry
- Status
- Active
- Wet-lab refresh
- Static
- Models reported
- 1
By benchmark (table view, 44 rows)
| Benchmark | Domain | Type | Evidence level | First release | Latest update | Status | Wet-lab refresh | Models reported | Current ranking |
|---|---|---|---|---|---|---|---|---|---|
| 01 CASP | Structure & co-folding | A | L4* | new round | Active | New | 15 | No ranking CASP has no single overall board and CASP17 results are not out yet. | |
| 05 PoseBusters | Structure & co-folding | C B | L3* | maintenance only | Active | Static | 8 | Not comparable 5 results on different protocols, no single ranking | |
| 15 CAMEO | Structure & co-folding | A | L4 | † | No sourced date† no later event | Active | New | 4 | Third-party No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the c… |
| 22 CAPRI | Structure & co-folding | A | L4* | † | new round | Round in progress | New | 2 | No ranking There is no ranking this round, and the entries come from the CASP16 assessment of the joint round rather tha… |
| 25 RNA-Puzzles | Structure & co-folding | A B | L2–L4* | † new round | Low-frequency | New | 3 | Board Kaggle private-board rank 1 team 0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10) Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles Board · Best site model: AlphaFold 3 | |
| 30 PLINDER / PINDER / Runs N' Poses | Structure & co-folding | C B | L3* | † | † new version | Active | Accumulating | 5 | Board success_rate 15.068; rmsd_success_rate 17.808; mean_lddt_pli 0.304; coverage 96.053 success_rate: share of predictions with LDDT-PLI > 0.8 and BiSyRMSD < 2; the page also lists rmsd_success_rate, mean_lddt_pli and coverage · Runs N' Poses (Polaris preview benchmark, plinder-org/runs-n-poses) Board · |
| 31 FoldBench | Structure & co-folding | B | L3* | board entry | Low-frequency | Static | 12 | Board Protein-Protein 72.93% / Antibody-Antigen 47.90% / Protein-Ligand 64.90%; Protein-RNA 62.32%, Protein-DNA 79.18%, RNA Monomer 0.61, DNA Monomer 0.53 Interface success rate (% ; DockQ>=0.23, protein-ligand requires LRMSD<2 A and LDDT-PLI>0.8); monomers use LDDT · FoldBench official repo leaderboard, full set (targets released after 2023-01) Board · | |
| 03 MoleculeNet (+OGB) | Small molecules | B | L2–L3* | paper revision | Frozen | Static | 2 | Not comparable 3 results on different protocols, no single ranking | |
| 04 TDC (ADMET + DTI) | Small molecules | B | L2–L3* | paper revision | Active | Static | 4 | Board None of the 22 official boards dates its entries, so every score is withheld and no entry can serve as headli… | |
| 06 CASF-2016 | Small molecules | B | L2* | no later sourced event found (as of 2026-11-16) | Frozen | Static | 0 | Not comparable 4 results on different protocols, no single ranking | |
| 07 USPTO family (+PaRoutes) | Small molecules | B | L2–L3* | data | Frozen | Static | 5 | Not comparable 5 results on different protocols, no single ranking | |
| 12 DUD-E / LIT-PCBA | Small molecules | B | L2* | † maintenance only | Low-frequency | Static | 2 | Third-party No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);… | |
| 19 PMO (Practical Molecular Optimization) | Small molecules | D | L2* | † new version | Low-frequency | Static | 2 | Self-reported Sum 18.362 (Table 3, 'Goal-directed hit generation results', 23 PMO tasks, AUC top-10) AUC top-10, summed over 23 oracles (max 23), 10,000-call budget · PMO 23 oracles (QED, DRD2, GSK3B, JNK3 plus 19 GuacaMol-derived), oracle functions via TDC Self-reported · stale not independently reproduced | |
| 20 CACHE | Small molecules | A | L5* | † | † data | Active | New | 0 | No ranking The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a… |
| 27 Polaris / ASAP / OpenADMET | Small molecules | A C | L4* | † new version | Low-frequency | New | 0 | No ranking Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET… | |
| 33 PoseX | Small molecules | B | L3* | paper revision | Active | Static | 6 | Third-party No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);… | |
| 44 FS-Mol | Small molecules | B | L3* | † archived | Frozen | Static | 0 | Not comparable 1 result on different protocols, no single ranking | |
| 02 ProteinGym | Proteins | B | L2–L3* | † | † data | Low-frequency | Accumulating | 10 | Board AIDO Protein-RAG (16B) 0.518 (SE 0.0) Average Spearman (zero-shot, DMS substitutions) · ProteinGym v1.3, 217 substitution DMS assays, zero-shot Board · stale Best site model: VenusREM (#2) |
| 17 CAFA | Proteins | A | L4–L5* | † | † new round | Active | Accumulating | 0 | No ranking CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline… |
| 26 FLIP / FLIP2 | Proteins | B | L2–L3* | † new version | Low-frequency | Static | 2 | Not comparable 3 results on different protocols, no single ranking | |
| 28 Adaptyv competitions / BenchBB | Proteins | A B | L5* | † | new round | Active | New | 4 | No ranking The current round has no results and the organiser page shows no ranks or scores. |
| 40 MotifBench | Proteins | D | L2* | board entry | Active | Static | 3 | Board RFD4-Proteina (indexed + medium thinking) 55.06 (± 1.35) MotifBench score (0-100) · 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline Board · Best site model: RFdiffusion3 (indexed) (#2) | |
| 34 AbBiBench | Antibodies | B | L2–L3* | paper revision | Active | Static | 6 | Board ProteinMPNN 0.29 Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays Board · Best site model: Boltz-2 (ipSAE) (#4) | |
| 35 FLAb / FLAb2 | Antibodies | B | L2–L3* | maintenance only | Active | Accumulating | 1 | Third-party The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is sho… | |
| 36 AIntibody | Antibodies | A | L5* | data | Low-frequency | New | 1 | Board AuraIDE / AuraBind (Aureka Biotechnologies, internal model) KD 94.7 pM (KinExA); best experimental antibody 113 pM Measured KD (HT-SPR and KinExA) plus five developability screens; placement per task · AIntibody round 1, 3 challenges, 511 designs from 29 organizations, SARS-CoV-2 spike RBD; all designs made and measured in one lab Board · | |
| 43 Ginkgo GDPa1 | Antibodies | A B | L4* | † | † data | Low-frequency | New | 0 | Board Anonymous 0.708 Spearman correlation per property on the private held-out set · AbDev competition 2025, 80 private held-out antibodies, 5 properties Board · |
| 08 Basenji/Enformer test set | Genomics & RNA | B | L2–L3* | † | † maintenance only | Active | Static | 3 | Self-reported The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given. |
| 11 ClinVar hold-out protocol | Genomics & RNA | B | L2–L3* | † | data | Active | Accumulating | 5 | Not comparable 5 results on different protocols, no single ranking |
| 13 CAGI | Genomics & RNA | A | L4–L5* | † | new round | Active | New | 1 | Third-party ClinPred / MetaRNN AUROC 0.92 AUROC (pathogenic vs benign missense) · CAGI6 Annotate-All-Missense full evaluation set (60 predictors) Third-party · stale |
| 16 GUE + Nucleotide Transformer benchmark | Genomics & RNA | B | L2–L3* | data | Low-frequency | Static | 0 | No ranking No official board and no site model reports on these suites, so there is no headline number. | |
| 21 DREAM5 / BEELINE / CausalBench | Genomics & RNA | A B C | L2–L5* | † | † paper revision | Low-frequency | Static | 0 | Not comparable 3 results on different protocols, no single ranking |
| 37 TraitGym | Genomics & RNA | B | L3* | † maintenance only | Low-frequency | Static | 3 | Board CADD 0.712 (se 0.038) AUPRC weighted-averaged by chromosome (main metric), with standard error · TraitGym Mendelian traits (matched_9) and Complex traits (matched_9), subset 'all', zero-shot and linear-probing tracks Board · stale Best site model: Evo2 40B (#5) | |
| 38 BEND | Genomics & RNA | B | L2* | new version | Active | Static | 0 | Board SpliceAI 0.674 Per-task metrics with no single official ranking metric: conditional log loss (bits/base, lower better), macro Spearman correlation, top-1 tissue identification accuracy · BEND 2.0 Table 1 snapshot: 8 endpoints across 5 capability categories, 17 model-interface rows Board · Best site model: AlphaGenome (#2) | |
| 39 RNAGym | Genomics & RNA | B | L2* | † board entry | Active | Static | 1 | Board 0.2120 (Ribozyme 0.1081 / tRNA 0.4310 / Aptamer 0.0970) Signed Spearman correlation, macro-averaged over three ncRNA categories with equal category weight (from v0.1.1) · RNAGym fitness leaderboard, ncRNA only (31 assays); mRNA coding/splicing assays excluded Board · † | |
| 09 scIB (+Open Problems) | Single-cell & perturbation | B C | L2* | new version | Active | Accumulating | 1 | Board The board orders by scaled overall mean but shows no numeric overall score; derived means are not used as a h… | |
| 24 Virtual Cell Challenge and perturbation benchmarks | Single-cell & perturbation | A B C | L2–L4* | † new round | Active | New | 9 | Board rank 1 of 300+ final-round teams (board publishes ranks only, no numeric scores) VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics · Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations) Board · | |
| 10 GuacaMol (+MOSES) | Molecule generation | D | L2* | maintenance only | Frozen | Static | 0 | No ranking No official board and no site model reports a GuacaMol score, so no headline is given. | |
| 32 CrossDocked2020 | Molecule generation | D | L2* | † | † data | Frozen | Static | 1 | Not comparable 3 results on different protocols, no single ranking |
| 14 PubMedQA | Text, LLMs & agents | B E | L1 | maintenance only | Frozen | Static | 2 | Board 82.0 Accuracy (%) (macro-F1 mostly not reported), reasoning-required setting · PQA-L test (500 questions) Board · stale | |
| 18 LAB-Bench / LABBench2 | Text, LLMs & agents | E | L1–L2* | maintenance only | Active | Static | 3 | Board 0.609 (Tools + high); Base 0.345 LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' · LABBench2 (1,912 tasks, 11 categories) Board · | |
| 23 BioASQ | Text, LLMs & agents | A B | L1–L4* | † | results | Active | Accumulating | 0 | Board mckpt2 Yes/No F1 0.9481, Acc. 0.9524; Factoid MRR 0.2000; List F1 0.1537 Yes/no macro-F1 as the ranking key of Tables 7 and 8, plus factoid MRR and list F1 · BioASQ Task 14b batch 2 exact answers, phase A+ (Table 7) and phase B (Table 8); preliminary results Board · † Best site model: MedQA-1 (university team; phase B uses a frontier-LLM ensemble) (#3) |
| 29 BixBench | Text, LLMs & agents | E | L2* | † | † board entry | Active | Static | 3 | Board 0.523 Average score over the successor's 20 tasks, scored programmatically against source papers · BixBench3 (successor benchmark, 20 research-study-scale tasks) Board · |
| 41 BioML-bench | Text, LLMs & agents | E | L2* | paper revision | Low-frequency | Static | 1 | Self-reported The source has no rank-1 entry: entries are unranked or protocol-specific, so the caveat is shown as for a no… | |
| 42 BioMysteryBench | Text, LLMs & agents | E | L2–L4* | † board entry | Active | Static | 1 | Third-party 83.70% Final-answer accuracy, mean of three full runs, scored by a judge model · Third-party board run: Terminus 2 harness, 200 turns / 4 h per task Third-party · † |
The ranking row shows only each benchmark's current first place; it cannot be compared across columns. Board = the organizer's leaderboard; self-reported = the developers' own report; third-party = an independent evaluation. † not fully verified (hover for the basis). * qualifier on the evidence level. Stale = snapshot more than 12 months old. Badges: board 18 · not comparable 9 · no ranking 8 · third-party 6 · self-reported 3.
- Of the 44 benchmarks, 11 have no established result from any of the 112 models, and 35 of the 112 models have none on the 44. Most public evidence for the newest models is wet-lab hit rate on self-chosen targets, not a shared test.
- Only 11 of 44 benchmarks produce fresh experimental data each round. The rest are static or grow from public records, so they cannot show whether a model works on a target nobody has measured yet.
- Of 196 recorded pairings, 131 are established: 38 official leaderboard or results, 31 developer reports on the official protocol, 29 on a modified protocol and 33 retrospective uses of released data. 10 are only cited as comparisons and 55 could not be confirmed.
- Only 7 of 112 models trained on wet-lab data their developers generated for the purpose. Most use public databases, and many lab-validated design models do not disclose their training data.
- Current rankings exist for few benchmarks: 18 have an organizer board, 6 only third-party and 3 only self-reported standings; 9 are not comparable across protocols and 8 have no ranking. 0 of 44 benchmarks reach L6, independent replication.
How to read this
A model paper's best result holds only on the benchmark it chose. Change the split, metric or time window and the position can differ. Rulers also differ: some truths are measured in blind experiments, others come from scoring functions; some test sets are well separated from training data, others leak widely. This page covers 44 widely used benchmarks and 112 models released in the past year, noting for each pairing whether it is author-reported, an official leaderboard or a third-party evaluation. It records what each benchmark tests, its evidence level and whether it is still usable. Scores appear only in each benchmark's own ranking, verbatim, with metric, source type and date, and are never compared across benchmarks.
Every pairing links one model to one benchmark and has one relation type and one source type (board, self-reported, third-party, or no confirmable source). Only established relations are counted as “reported”.
| Relation type | Meaning | Pairings | Counted |
|---|---|---|---|
| Official leaderboard or results | organizer results or leaderboard under an identifiable name (counted, established) | 38 | yes |
| Developer report, official protocol | developer reports on the official split/protocol, not on a board (counted, established) | 31 | yes |
| Developer report, modified protocol | developer reports with different filtering/subset/sampling, or self-reported participation the organizer has not confirmed (counted, established) | 29 | yes |
| Third-party evaluation | only third parties ran the model on the benchmark (ADDED this round by owner decision; counted, established) | 0 | yes |
| Retrospective dataset use | post-hoc reuse of a released round's data (established under this round's definition, but shown separately and never as participation) | 33 | yes |
| Cited as comparison only | the model appears only as a baseline number copied from elsewhere, or only the tool is used (NOT established) | 10 | no |
| Unconfirmed | claimed or second-hand, not verifiable (NOT established) | 55 | no |
Evidence levels
Levels describe the type of evidence a benchmark gives, not how good it is. A benchmark can span several, and scores on different levels answer different questions: a 0.9 correlation on L2 and a 10% hit rate on L5 are not comparable.
| Level | Meaning |
|---|---|
| L1 | Knowledge recall (question answering, multiple choice) |
| L2 | Retrospective random or in-distribution split |
| L3 | Stratified or out-of-distribution split (similarity, scaffold, time) |
| L4 | Blind test with hidden truth (truth does not exist at submission) |
| L5 | Prospective wet-lab experiment |
| L6 | Independent replication |
Benchmark types
| Type | Meaning | Examples |
|---|---|---|
| A | Periodic blind challenge; new questions and new truth each round | CASP, CAPRI, CAGI, CACHE, AIntibody |
| B | Static dataset suite; download and run | ProteinGym, MoleculeNet, TDC, FoldBench |
| C | Infrastructure: validators, similarity-split tools, evaluation protocols | PoseBusters checker, PLINDER/PINDER splits |
| D | Computational-oracle benchmark; truth comes from a scoring function or fold read-back | GuacaMol, PMO, CrossDocked2020, MotifBench |
| E | Task environments for agents and LLMs | LAB-Bench, BixBench, BioML-bench, BioMysteryBench |
Ten findings
- Only a few blind tests can show methodological progress12 of 44 benchmarks reach L4 (blind tests with hidden truth) and 6 reach L5 (prospective wet-lab experiments). CASP, CAPRI, CAFA, CAGI, CAMEO, CACHE and AIntibody are the long-running or recurring ones. For the rest, isolation and generalization controls are weaker.
- The most influential benchmarks are often the weakestMoleculeNet, USPTO, CASF-2016, DUD-E and GuacaMol owe part of their results to memorization and constructed shortcuts. Static sets that can be tuned indefinitely are the ones that accumulate influence.
- Simple baselines repeatedly match deep modelsRetrieval on CASF-2016, XGBoost on a TDC task, ECFP against pretrained embeddings, PrediXcan against Enformer, additive or linear baselines in perturbation, randomly initialized DNA models, template retrieval in RNA 3D, and alignment-based methods on TraitGym.
- Metric validity is a field-wide weaknessMost benchmarks report no confidence intervals, and gaps among leading entries are often within sampling noise. PubMedQA, PoseBusters V2 and several FoldBench tasks have small test sets; FoldBench's authors advise against interpreting small gaps.
- Leakage now means similarity must be first-classMany CASF-2016 test complexes have near-duplicates in training, and most USPTO-50K test reactions match training templates. Under PLINDER and FoldBench, co-folding models degrade in low-similarity bins, so later-cutoff models should report later subsets.
- Antibody evaluation is new, and early findings are negativeAll 4 antibody benchmarks first appeared in 2024 or later. Results do not transfer across tasks, cross-validation exceeds held-out performance, zero-shot correlation is often non-significant, and specialized fine-tuning is not reliably helpful. Most antibody-specific models report only on self-built sets.
- Newer benchmarks are almost all staticOf the 14 added entries, 12 produce no new experimental data each round; CrossDocked2020 and MotifBench use docking scores and fold read-back as truth. Only FoldBench is widely adopted: maintainer-run leaderboard, antibody-antigen coverage, no waiting.
- Official leaderboards are mostly run by maintainers52% of official-leaderboard entries sit on ProteinGym, FoldBench and AbBiBench, all run by the benchmark owners. Genuine submission-based boards are few, such as MotifBench and LABBench2/BixBench3.
- Agent and LLM boards exist, but version control is a new problemLABBench2 and BixBench3 have dated official leaderboards, an improvement on self-reporting. BixBench3 differs from BixBench; vendor system cards restate sub-items inconsistently; BioMysteryBench is vendor-authored with a non-public full set; PubMedQA is saturated.
- Infrastructure is the most underrated contributionPoseBusters (validity checks beyond RMSD), PLINDER (similarity-stratified splits), PMO (fixed oracle budget), Polaris (bootstrap and multiple-comparison statistics) and MotifBench (generate-and-refold pipeline). No benchmark provides quantitative evidence that better rankings raise wet-lab hit rates.
Who reports what
The matrix of 112 models against 44 benchmarks is mostly empty. Figure 1 shows how thin the coverage is: only 5 benchmarks are reported by eight or more models (CASP 15, FoldBench 12, ProteinGym 10, Virtual Cell Challenge and perturbation benchmarks 9, PoseBusters 8), and 11 by none. Figure 2 shows the other side, how few benchmarks a typical model reports. Figure 3 asks who vouches for the numbers.
Figure 1. Which benchmarks do models actually report?
Models with an established result on each of the 44 benchmarks
View as table
| Benchmark | Set | Models counted | By relation type |
|---|---|---|---|
| 01 CASP | Core | 15 | board 3, retrospective 12, unconfirmed 1 |
| 31 FoldBench | Supplementary | 12 | board 7, self 3, modified protocol 2 |
| 02 ProteinGym | Core | 10 | board 7, self 2, retrospective 1 |
| 24 Virtual Cell Challenge and perturbation benchmarks | Core | 9 | board 1, modified protocol 6, retrospective 2 |
| 05 PoseBusters | Core | 8 | self 4, modified protocol 4, cited only 3 |
| 33 PoseX | Supplementary | 6 | retrospective 6 |
| 34 AbBiBench | Supplementary | 6 | board 6 |
| 07 USPTO family (+PaRoutes) | Core | 5 | self 4, modified protocol 1 |
| 11 ClinVar hold-out protocol | Core | 5 | modified protocol 5 |
| 30 PLINDER / PINDER / Runs N' Poses | Core | 5 | self 3, modified protocol 2, cited only 4 |
| 04 TDC (ADMET + DTI) | Core | 4 | board 3, self 1, unconfirmed 2 |
| 15 CAMEO | Core | 4 | board 1, retrospective 3, unconfirmed 1 |
| 28 Adaptyv competitions / BenchBB | Core | 4 | board 2, modified protocol 1, retrospective 1 |
| 08 Basenji/Enformer test set | Core | 3 | self 3 |
| 18 LAB-Bench / LABBench2 | Core | 3 | board 1, modified protocol 2, unconfirmed 5 |
| 25 RNA-Puzzles | Core | 3 | modified protocol 2, retrospective 1, cited only 1 |
| 29 BixBench | Core | 3 | board 1, self 1, modified protocol 1, unconfirmed 6 |
| 37 TraitGym | Supplementary | 3 | modified protocol 1, retrospective 2 |
| 40 MotifBench | Supplementary | 3 | board 3 |
| 03 MoleculeNet (+OGB) | Core | 2 | self 1, modified protocol 1, unconfirmed 1 |
| 12 DUD-E / LIT-PCBA | Core | 2 | retrospective 2, unconfirmed 3 |
| 14 PubMedQA | Core | 2 | self 2, unconfirmed 6 |
| 19 PMO (Practical Molecular Optimization) | Core | 2 | self 1, modified protocol 1, cited only 1 |
| 22 CAPRI | Core | 2 | board 2, unconfirmed 1 |
| 26 FLIP / FLIP2 | Core | 2 | self 2 |
| 09 scIB (+Open Problems) | Core | 1 | self 1, unconfirmed 2 |
| 13 CAGI | Core | 1 | retrospective 1 |
| 32 CrossDocked2020 | Supplementary | 1 | self 1 |
| 35 FLAb / FLAb2 | Supplementary | 1 | retrospective 1 |
| 36 AIntibody | Supplementary | 1 | board 1, unconfirmed 1 |
| 39 RNAGym | Supplementary | 1 | self 1, unconfirmed 1 |
| 41 BioML-bench | Supplementary | 1 | retrospective 1, unconfirmed 6 |
| 42 BioMysteryBench | Supplementary | 1 | self 1, unconfirmed 6 |
| 06 CASF-2016 | Core | 0 | none |
| 10 GuacaMol (+MOSES) | Core | 0 | none |
| 16 GUE + Nucleotide Transformer benchmark | Core | 0 | none |
| 17 CAFA | Core | 0 | unconfirmed 1 |
| 20 CACHE | Core | 0 | none |
| 21 DREAM5 / BEELINE / CausalBench | Core | 0 | unconfirmed 2 |
| 23 BioASQ | Core | 0 | unconfirmed 8 |
| 27 Polaris / ASAP / OpenADMET | Core | 0 | cited only 1, unconfirmed 2 |
| 38 BEND | Supplementary | 0 | none |
| 43 Ginkgo GDPa1 | Supplementary | 0 | none |
| 44 FS-Mol | Supplementary | 0 | none |
Figure 2. Most models report on zero or one benchmark
Number of the 112 models, by how many of the 44 benchmarks they have an established result on
View as table
| Benchmarks per model | Models |
|---|---|
| 0 | 35 |
| 1 | 48 |
| 2 | 15 |
| 3 | 7 |
| 4 | 4 |
| 5 or more | 3 |
Figure 3. Who vouches for the number?
All 196 recorded pairings by source type, grouped by model area
View as table
| Model area | Self-reported | Third-party | Official leaderboard | Unconfirmed source |
|---|---|---|---|---|
| Structure & co-folding | 16 | 24 | 16 | 6 |
| Small molecules | 13 | 5 | 3 | 6 |
| Proteins | 5 | 7 | 15 | 0 |
| Antibodies | 0 | 0 | 1 | 0 |
| Genomics & RNA | 10 | 3 | 0 | 1 |
| Single-cell & perturbation | 7 | 2 | 1 | 4 |
| Molecule generation | 1 | 0 | 0 | 0 |
| Text, LLMs & agents | 8 | 2 | 2 | 38 |
The full model-by-benchmark matrix, with hover details for each of the 196 recorded pairings, is on the explorer page.
- 19 of 44 benchmarks are reported by at most one model, and 11 by none: CASF-2016, GuacaMol, GUE + Nucleotide Transformer benchmark, CAFA, CACHE, DREAM5 / BEELINE / CausalBench, BioASQ, Polaris / ASAP / OpenADMET, BEND, Ginkgo GDPa1, FS-Mol. Older ones are seen as undiscriminating or costly to enter; the newest have not yet been adopted.
- Among the newer 14 entries, only FoldBench is genuinely adopted: maintainer-run leaderboard, antibody-antigen interfaces, no waiting for a window. All AbBiBench entries were registered by its own maintainers; most others have one to three entries.
- Most models report on zero or one benchmark. Those with zero are mostly protein-design and agent systems whose public results are wet-lab hit rates on self-selected targets or internal metrics, with no shared yardstick.
- Public evaluation coverage is inversely related to commercialization. Structure and co-folding have several shared yardsticks and a full matrix; protein design and antibody models, whose unit of result is targets and experimental batches, leave it sparse.
- Unconfirmed pairings are concentrated among agent and LLM systems, whose official materials do not report these benchmarks although external lists and media fill them in. Other cases are disputes over provenance or anonymized official entries.
- Official leaderboards are a minority and unevenly distributed. ProteinGym, FoldBench and AbBiBench account for 52% of them, all run by maintainers. Most leaderboard results therefore reflect maintainer willingness to evaluate, not industry submission habits.
How strong is the evidence?
A benchmark is only as informative as the question it can answer. Figure 4 counts the benchmarks that reach each evidence level; Figure 6 shows when they appeared and whether anyone still maintains them.
Figure 4. Evidence level: none reaches independent replication
Benchmarks that touch each evidence level (a benchmark can span several levels)
View as table
| Level | Meaning | Benchmarks touching it |
|---|---|---|
| L1 | Knowledge recall (question answering, multiple choice) | 3 |
| L2 | Retrospective random or in-distribution split | 27 |
| L3 | Stratified or out-of-distribution split (similarity, scaffold, time) | 21 |
| L4 | Blind test with hidden truth (truth does not exist at submission) | 12 |
| L5 | Prospective wet-lab experiment | 6 |
| L6 | Independent replication | 0 |
Figure 6. Many benchmarks are young, and some are no longer maintained
Each dot is one benchmark, placed by the year of its first sourced release
View as table
| First release | Benchmark | Status |
|---|---|---|
| 1994 | 01 CASP | Active |
| 2001 (partly verified) | 22 CAPRI | Round in progress |
| 2009-04-09 | 06 CASF-2016 | Frozen |
| 2010 (partly verified) | 13 CAGI | Active |
| 2010-09 (partly verified) | 17 CAFA | Active |
| 2010 (partly verified) | 21 DREAM5 / BEELINE / CausalBench | Low-frequency |
| 2012-07-05 | 12 DUD-E / LIT-PCBA | Low-frequency |
| 2012 (partly verified) | 15 CAMEO | Active |
| 2012-02-23 | 25 RNA-Puzzles | Low-frequency |
| 2013-04 (partly verified) | 11 ClinVar hold-out protocol | Active |
| 2013 (partly verified) | 23 BioASQ | Active |
| 2014-02 | 07 USPTO family (+PaRoutes) | Frozen |
| 2017-03-02 | 03 MoleculeNet (+OGB) | Frozen |
| 2017-07-10 (partly verified) | 08 Basenji/Enformer test set | Active |
| 2018-11-22 | 10 GuacaMol (+MOSES) | Frozen |
| 2019-09-13 | 14 PubMedQA | Frozen |
| 2020-11-13 | 04 TDC (ADMET + DTI) | Active |
| 2020-05-23 | 09 scIB (+Open Problems) | Active |
| 2020-02-20 (partly verified) | 32 CrossDocked2020 | Frozen |
| 2021-12-01 (partly verified) | 20 CACHE | Active |
| 2021-11-11 | 26 FLIP / FLIP2 | Low-frequency |
| 2021-12 | 44 FS-Mol | Frozen |
| 2022-05-27 (partly verified) | 02 ProteinGym | Low-frequency |
| 2022-06-10 | 16 GUE + Nucleotide Transformer benchmark | Low-frequency |
| 2022-06-22 | 19 PMO (Practical Molecular Optimization) | Low-frequency |
| 2023-08-10 | 05 PoseBusters | Active |
| 2023-11-23 | 27 Polaris / ASAP / OpenADMET | Low-frequency |
| 2023-11-21 | 38 BEND | Active |
| 2024-07-14 | 18 LAB-Bench / LABBench2 | Active |
| 2024-08-20 | 24 Virtual Cell Challenge and perturbation benchmarks | Active |
| 2024-07-18 (unverified) | 28 Adaptyv competitions / BenchBB | Active |
| 2024-07-17 (partly verified) | 30 PLINDER / PINDER / Runs N' Poses | Active |
| 2024-01-15 | 35 FLAb / FLAb2 | Active |
| 2024-11-04 | 36 AIntibody | Low-frequency |
| 2025-02-28 (partly verified) | 29 BixBench | Active |
| 2025-05-27 | 31 FoldBench | Low-frequency |
| 2025-05-03 | 33 PoseX | Active |
| 2025-05-23 | 34 AbBiBench | Active |
| 2025-02-12 | 37 TraitGym | Low-frequency |
| 2025-06-17 | 39 RNAGym | Active |
| 2025-02-18 | 40 MotifBench | Active |
| 2025-09-04 | 41 BioML-bench | Low-frequency |
| 2025-09-08 (partly verified) | 43 Ginkgo GDPa1 | Low-frequency |
| 2026-04-29 | 42 BioMysteryBench | Active |
Where new wet-lab data come from
A benchmark that never receives new experimental data can only tell you how a model does on what is already known. Figure 5 sorts the 44 benchmarks by how they refresh. Among the 14 supplementary entries, only 2 generate fresh experiments.
Figure 5. Where do new wet-lab data come from?
Benchmarks by how they get new data, grouped by area
View as table
| Area | Fresh experiments | Grows from public sources | Static |
|---|---|---|---|
| Structure & co-folding | 4 | 1 | 2 |
| Small molecules | 2 | 0 | 8 |
| Proteins | 1 | 2 | 2 |
| Antibodies | 2 | 1 | 1 |
| Genomics & RNA | 1 | 1 | 6 |
| Single-cell & perturbation | 1 | 1 | 0 |
| Molecule generation | 0 | 0 | 2 |
| Text, LLMs & agents | 0 | 1 | 5 |
A. Periodically generate new experimental data (blind or prospective)
Organizers or partners produce new measurements each round: unpublished structures, functional experiments, compounds synthesized and assayed, new Perturb-seq, or designs tested in a single lab. Some are one-off or non-periodic.
Examples: CASP, CAMEO, CAPRI, RNA-Puzzles, CAGI, CACHE, Polaris/OpenADMET, Adaptyv/BenchBB, AIntibody, GDPa1, Virtual Cell Challenge
B. Updated periodically, but not new experiments
New data come from time-delayed accumulation of historical annotations, clinical submissions, weekly PDB updates, newly written expert questions, new question versions on existing sequencing data, or community-contributed published assays.
Examples: CAFA, ClinVar protocol, PLINDER/PINDER, BioASQ, LABBench2, BioMysteryBench, FLAb/FLAb2
C. Static (reused historical public data or computational oracles)
No new data are added. ProteinGym occasionally incorporates new DMS data; AbBiBench ran wet-lab work once.
Examples: MoleculeNet, TDC, PoseBusters, CASF, USPTO, Enformer, scIB, GuacaMol, DUD-E/LIT-PCBA, PubMedQA, GUE/NT, PMO, DREAM5/BEELINE, FLIP, BixBench, FoldBench, CrossDocked2020, PoseX, TraitGym, BEND, RNAGym, MotifBench, BioML-bench, FS-Mol
Do you need your own wet-lab data to evaluate a new model?
- Static (C) benchmarks need no new wet lab and can be run by download. The cost is retrospective conclusions only, and several leak (CASF, USPTO, DUD-E); the later a model's training cutoff, the worse the leakage.
- Blind (A) benchmarks (CASP, CAPRI, CAMEO, CAGI, CACHE, Polaris, VCC) need no self-generated data but require waiting for the next prediction window: weekly for CAMEO, every two years for CASP. Released rounds can be run as static sets, but are no longer blind for models with later training cutoffs.
- Own wet-lab work is needed for protein design (for example Adaptyv-style per-sequence testing, a few dozen to a few hundred sequences per round), for molecule generation and optimization (GuacaMol and PMO have only computational oracles, so real activity requires synthesis and assay), and when conclusions must generalize to own targets or chemical space (true for TDC and Polaris).
What the models are trained on
Every model page records four kinds of training data: public databases, private or licensed data, wet-lab data the developers generated specifically for training, and wet-lab work used only to validate outputs. Figure 7 assigns each model to the first group that applies.
Figure 7. What are the models trained on?
Models by training-data provenance (each model counted once, in the highest-effort group that applies)
View as table
| Model area | Public data only | Public data + wet lab for validation only | Private or licensed data | Own wet-lab training data | Not disclosed |
|---|---|---|---|---|---|
| Structure & co-folding | 17 | 4 | 1 | 0 | 2 |
| Small molecules | 14 | 2 | 4 | 1 | 0 |
| Proteins | 9 | 14 | 1 | 2 | 0 |
| Antibodies | 0 | 6 | 0 | 1 | 0 |
| Genomics & RNA | 7 | 2 | 0 | 0 | 0 |
| Single-cell & perturbation | 7 | 2 | 0 | 2 | 1 |
| Molecule generation | 1 | 0 | 0 | 0 | 0 |
| Text, LLMs & agents | 4 | 4 | 1 | 1 | 2 |
- Only 7 of 112 models have self-generated wet-lab training data: Enchant v1/v2, EVOLVEpro, HCAbLM, ProGen3, Tahoe-x1, VERID, X-Cell. Origin-1 and Dyno Psi-1 are described by their companies as using experimental data, but this is unconfirmed in papers or released weights.
- Using third-party wet-lab data is not self-generation. STATE, Stack, Lingshu-Cell and xTrimoSCPerturb/scFoundation use perturbation data from Tahoe, Parse or Arc VCC and do not produce the data themselves.
- 7 of 112 models record private or licensed training data: Beacon (Inductive Bio), Chai-3, Chemformer, KERMT, MedGemma / Med-Gemini, OpenFold3, Uni-Mol3.
- Wet lab used only to validate outputs is recorded for 37 models (34 of them in the validation-only group of Figure 7; the rest also have private or own wet-lab data), most in protein design. Apart from ProGen3, HCAbLM and VERID, none claim the results were fed back into training.
- 53 of 112 models document public data only; 33 more use public data and do not disclose whether they also use private data. Many public sources already contain distilled or synthetic data, and 5 models disclose no training data at all.
- Of the 7 antibody models, HCAbLM (a transgenic-mouse immune repertoire) is the only one with self-generated training data; 7 report wet-lab validation.
- Newer models add computationally self-generated data (MD trajectories, generated reactions), which is not wet lab, and synthetic interface data now common in protein design. VERID is a new form: agent-loop online feedback.
- LLM and agent models are not trained on wet-lab data and have sparse validation. Several have none; Biomni and C2S-Scale have very small validations, and Co-Scientist and Robin were validated by partner labs.
Choosing a benchmark set
No single benchmark answers a research question. Each scenario below pairs two to four complementary benchmarks, says what each tests and why it was chosen, which variables to control, and what the set can and cannot support.
S1 · Protein monomer and complex structure
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| CASP Prospective structure prediction by track | Blind: truth does not exist at submission. | Training cutoff vs target release date; template/MSA use; top-1 vs top-k; manual intervention. |
| CAMEO Rolling weekly structure prediction | Shows stability rather than peak performance. | Evaluation window; training cutoff. |
| CAPRI Protein complex interface quality | Source of the DockQ criterion. | Sampling count; manual intervention. |
| FoldBench Monomer and interface tasks for cross-model comparison | Maintainer-run leaderboard; no waiting for a window. | Small RNA/DNA target counts; training cutoff. |
The set can support: Structure generalization within a stated round.
It cannot support: Not functional conclusions or drug-design usability.
S2 · Co-folding and ligand pose
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| PoseBusters V2 Geometric and physical validity of ligand poses | The 18-check validator is its main legacy. | Version (V1/V2, SD/CD); training cutoff; pose-selection rule. |
| Runs N' Poses (PLINDER) Docking and co-folding by similarity bin | Makes similarity stratification first-class. | Report nearest-neighbour training-pose copy and physics docking baselines. |
| PoseX Self-docking and cross-docking scored separately | Compares physics, AI docking and co-folding under one protocol. | Report chirality and clash errors separately. |
| FoldBench protein-ligand subset Protein-ligand structure | Independent check of similarity effects. | Training cutoff; later-cutoff models need late-subset reporting. |
The set can support: Pose accuracy by similarity bin; PLINDER-PL50 when training own models.
It cannot support: Not affinity conclusions.
S3 · Affinity scoring and virtual screening
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| CASF-2016 Historical affinity scoring | Alignment with the literature only. | Similarity-retrieval baseline; leakage-cleaned re-tests (LP-PDBBind, CleanSplit). |
| LIT-PCBA Virtual screening after de-duplication | Less leakage-prone than DUD-E. | Molecule counting unit after de-duplication. |
| CACHE or Polaris blinds Prospective hit finding / blind prediction | Truth generated after submission. | Library-level overlap of training and evaluation sets (ChEMBL, BindingDB). |
The set can support: Relative gain after leakage removal and qualitative prospective-hit evidence.
It cannot support: Leaderboard position is not evidence of screening utility.
S4 · Variant effects and protein engineering
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| ProteinGym Zero-shot fitness and variant-effect ranking | Community default for protein language models. | Stratify by assay type; report multiple metrics; separate leaderboard from self-chosen DMS subsets. |
| FLIP/FLIP2 Extrapolation splits | Tests ability along defined extrapolation axes. | Training recipe and random seed (some landscapes vary widely). |
| CAGI MAVE challenges Blind variant-effect prediction | Truth unpublished at submission. | MSA depth and taxon; viral proteins reported separately. |
The set can support: Within-assay ranking and ability along stated extrapolation axes.
It cannot support: Not a claim of best engineering model.
S5 · Variant pathogenicity
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| ClinVar Time- or gene-held-out pathogenicity classification | Retrospective clinical labels. | Label circularity with training features; gene-level label imbalance. |
| ProteinGym clinical track Clinical variant effect | Common reference point. | Separate zero-shot from supervised probes (conclusions differ for genomic models such as Evo 2). |
| CAGI clinical challenges Blind clinical prediction | Only source of blind evidence here. | Challenge-specific protocol. |
The set can support: Retrospective classification ability; blind evidence only from CAGI.
It cannot support: Retrospective labels cannot stand in for prospective validation.
S6 · Protein binder and de novo design
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| Adaptyv/BenchBB Binder design rounds measured in one lab | Wet-lab truth on a shared platform. | Hit definition; designs per target disclosed; target-difficulty strata. |
| Own or paid prospective SPR/BLI Binding of own designs | Answers whether designs work. | Do not screen on interface-confidence metrics alone. |
| MotifBench Motif scaffolding | Unified generate-and-refold pipeline. | Truth is folding read-back, not experiment. |
The set can support: Qualitative comparison of hit rates on specified targets.
It cannot support: A single round cannot be used to rank methods.
S7 · Antibody design and developability
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| AIntibody Prospective antibody design blind | Only prospective blind in this area; next round pending. | Same-lab measurements; one round per edition. |
| GDPa1 Held-out developability prediction | Held-out blind exposes cross-validation optimism. | Report cross-validation and held-out results side by side. |
| AbBiBench Zero-shot score vs measured binding | Compares structure-conditioned and antibody-specific models. | Do not mix with self-defined antibody-antigen subsets. |
| FLAb2 Seven property classes | Zero-shot correlation across assays. | Germline similarity as covariate. |
| FoldBench antibody-antigen Interface structure | Structure only. | Keep structure accuracy separate from functional prediction. |
The set can support: Strength of zero-shot correlation with measurements; developability error on held-out sets.
It cannot support: Not that antibody design is ready for use.
S8 · Protein function annotation
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| CAFA Time-delayed GO annotation, no-knowledge subset | Prospective, time-split. | Report Naive and BLAST baselines; GO version and evidence filters; per sub-ontology. |
| Own time-split set Temporal generalization | Self-built check. | A self-chosen GO test set is not a CAFA blind; separate text-mining from sequence-only methods. |
The set can support: Temporal generalization.
It cannot support: Unannotated does not mean wrong.
S9 · ADMET and few-shot new assays
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| TDC-ADMET ADMET property prediction | Common reference; use multiple seeds. | Strong baselines such as XGBoost with ECFP; hold out by lab source and time. |
| Polaris/OpenADMET blinds Blind ADMET prediction | Unpublished real project data. | Winning entries are often anonymized and cannot be claimed. |
| FS-Mol Few-label adaptation with support-set budget curves | Low-label new-assay setting. | Target-family isolation; binarization threshold stated. |
| Own time-forward backtest Project-specific error | Reflects the actual chemical series. | MoleculeNet for historical alignment only. |
The set can support: Practical error on a specified chemical series; adaptation efficiency under few labels.
It cannot support: Leaderboard position is not a basis for R&D decisions.
S10 · Molecule generation and optimization
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| PMO Sample-efficient optimization under a fixed oracle budget | Budget curves reveal search efficiency. | Oracle call count, total compute, multiple seeds; report unconstrained baselines. |
| GuacaMol Distribution and goal-directed tasks | Smoke test only. | Most PMO oracles come from GuacaMol, so they are not independent evidence. |
| Chemistry-quality and synthesizability filters Quality of generated molecules | Guards against shortcut scores. | Apply before reporting scores. |
The set can support: Search efficiency within a budget.
It cannot support: Not real activity.
S11 · Structure-conditioned molecule generation
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| CrossDocked2020 Pocket-conditioned generation | Standard split. | State the split (original dataset and generation subset differ). |
| PoseBusters Validity and interaction recovery | Physical plausibility. | Report drug-likeness and synthesizability, not only docking score. |
| Own pockets Docking and synthesis assessment | Tests the intended target. | Report pocket similarity to training set; docking score is an oracle, not truth. |
The set can support: Geometry and docking behaviour of generated molecules in given pockets.
It cannot support: Not activity or druggability.
S12 · Retrosynthesis and reaction prediction
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| USPTO-50K/MIT Patent-text reaction prediction | Alignment only. | Same-patent reactions can cross splits; template-retrieval baseline. |
| PaRoutes Route-level evaluation | Beyond single steps. | Round-trip metrics depend on a forward model and carry proxy bias. |
The set can support: Reproduction at the patent-text level.
It cannot support: Whether something can be made needs synthesis validation.
S13 · DNA and regulatory genomics
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| GUE/NT (revised) DNA language model tasks | Smoke test with multiple seeds. | Composition-matched negatives; chromosome hold-out; small CNN baseline. |
| BEND Functional tasks | Broader task set. | Do not mix its two versions. |
| Enformer/Borzoi Expression prediction | Report across-gene and across-individual separately. | PrediXcan-type and alignment-based baselines. |
| TraitGym Causal non-coding variants | Matched controls. | Using its variant set is not the same as being on its leaderboard. |
The set can support: Representation quality, reference-genome fit and causal-variant identification under matched controls.
It cannot support: Individual-level effects need separate validation.
S14 · RNA
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| RNA-Puzzles, CASP RNA, Kaggle rounds RNA 3D blind prediction | Only blind tradition for RNA. | Template stratification; family leakage. |
| RNAGym Fitness and structure tasks | Broader RNA tasks. | State the source of scale figures. |
| Secondary-structure sets 2D structure | Split by family. | A self-built set containing some RNA-Puzzles targets is not an RNA-Puzzles result. |
The set can support: 3D versus 2D and structure versus fitness conclusions, each independent.
It cannot support: They cannot substitute for one another.
S15 · Single-cell integration, perturbation and regulatory networks
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| scIB Batch integration | Run zero-shot and fitted settings. | Baselines: HVG plus PCA, scVI. |
| PerturBench Covariate transfer in perturbation prediction | Tests transfer. | Resample at donor and batch level. |
| VCC Perturbation prediction, multiple metrics | Prospective annual challenge. | Mean, no-change, additive and linear baselines; separate post-hoc evaluation from entries. |
| DREAM5/BEELINE Network inference | Per-network comparison. | Report per network, do not aggregate; unobserved edges are not negatives. |
The set can support: Gain relative to this baseline set.
It cannot support: Not that a virtual cell is usable.
S16 · Literature assistants, bioinformatics and research agents
| Benchmark and what it tests | Why it is in the set | Variables to control |
|---|---|---|
| BioASQ Literature question answering | Use the current batch. | Version and date. |
| LABBench2 Lab-style tasks, with and without tools | Official leaderboard with update date. | Tool access; token and time budgets; grader model. |
| BixBench Bioinformatics analysis, full set and subsets | Report both. | Original and successor question banks must not be listed together. |
| BioML-bench End-to-end ML pipelines | Pipeline-level test. | Human baseline as lower or upper bound. |
| BioMysteryBench Mystery-style analysis tasks | Questions written by a model vendor; full set not public. | Separate vendor-run from third-party reruns. |
The set can support: Trend in gains from tools.
It cannot support: Not the absolute level of scientific ability; cite the official leaderboard update date.
Entity pages
Every benchmark and every model has its own page built from one record with the same fields: dates with their basis and verification, relation types for each pairing, the current ranking, training data in four classes, wet-lab validation with numerators and denominators, sources and corrections.
Data, method and limits
Compiled from public papers, official repositories and leaderboards, and checked in multiple independent verification rounds. No benchmark was run for this survey, and no self-measured results appear. Conflicting claims are marked unconfirmed; items not found in public sources are marked N/A (not public). Vendor press releases, single non-peer-reviewed preprint critiques and third-party aggregator figures are down-weighted and used only as orders of magnitude. Scores are shown only inside each benchmark's ranking, copied verbatim with metric, eval set, source type and date; values that could not be verified are shown as unverified and not as numbers. Results not yet published for 2026 events (CASP17, CAFA6, VCC 2026, the next AIntibody) are excluded. Snapshot date: 2026-11-16.
- Pairings come from one relation file (196 rows). Benchmark pages, model pages, the overview table, the figures and the explorer are all generated from it, so their counts agree.
- Scores appear only in rankings, copied verbatim with metric, eval set, source type and date. Not-comparable benchmarks show no headline score; values that could not be verified, or that lack a source link or date, are withheld.
- Company press releases and single non-peer-reviewed preprints are treated as weak evidence and flagged where they occur. † marks a date or value that is not fully verified.
- Fast-moving items (CASP17 results, CAFA6, Virtual Cell Challenge 2026, the next AIntibody round) were not published at the snapshot date and are excluded.
Compiled from public papers, official repositories and leaderboards; every entity page lists its sources. No benchmark was run for this survey. Snapshot 16 Nov 2026.