AI for Bio · Benchmarks

AI for Bio benchmarks: what 44 tests can and cannot tell you

A survey of 44 widely used AI-for-biology benchmarks and the 112 models released or updated in the last year that are measured on them: what each test measures, which models actually report it, whether new wet-lab data keeps flowing in, and where the training data comes from. Every benchmark and model has its own page. Scores appear only in each benchmark's ranking, verbatim and never across benchmarks.

Spark Science ResearchSnapshot 16 Nov 202616 min read

The 44 benchmarks: identity, freshness and current ranking

One column per benchmark. The last row is each benchmark's current ranking; metrics and protocols differ by column, so scores cannot be compared across columns. As of 2026-11-16; each cell shows its own snapshot date.

Skip to the by-benchmark table
CSV JSON
The 44 benchmarks: identity, freshness and current ranking. One column per benchmark. The last row is each benchmark's current ranking; metrics and protocols differ by column, so scores cannot be compared across columns. As of 2026-11-16; each cell shows its own snapshot date.
DomainStructure & co-foldingSmall moleculesProteinsAntibodiesGenomics & RNASingle-cell & perturbationMolecule generationText, LLMs & agents
Benchmark01 CASP05 PoseBusters15 CAMEO22 CAPRI25 RNA-Puzzles30 PLINDER / PINDER / Runs N' Poses family31 FoldBench03 MoleculeNet (+OGB)04 TDC (ADMET + DTI) family06 CASF-201607 USPTO family (+PaRoutes) family12 DUD-E / LIT-PCBA family19 PMO (Practical Molecular Optimization)20 CACHE27 Polaris / ASAP / OpenADMET33 PoseX44 FS-Mol02 ProteinGym family17 CAFA26 FLIP / FLIP2 family28 Adaptyv competitions / BenchBB family40 MotifBench34 AbBiBench35 FLAb / FLAb236 AIntibody43 Ginkgo GDPa108 Basenji/Enformer test set family11 ClinVar hold-out protocol13 CAGI16 GUE + Nucleotide Transformer benchmark family21 DREAM5 / BEELINE / CausalBench family37 TraitGym38 BEND39 RNAGym09 scIB (+Open Problems) family24 Virtual Cell Challenge and perturbation benchmarks family10 GuacaMol (+MOSES) family32 CrossDocked2020 family14 PubMedQA18 LAB-Bench / LABBench2 family23 BioASQ29 BixBench41 BioML-bench42 BioMysteryBench
TypeAC BAAA BC BBBBBBBDAA CBBBABA BDBBAA BBBABA B CBBBB CA B CDDB EEA BEEE
Evidence levelL4*L3*L4L4*L2–L4*L3*L3*L2–L3*L2–L3*L2*L2–L3*L2*L2*L5*L4*L3*L3*L2–L3*L4–L5*L2–L3*L5*L2*L2–L3*L2–L3*L5*L4*L2–L3*L2–L3*L4–L5*L2–L3*L2–L5*L3*L2*L2*L2*L2–L4*L2*L2*L1L1–L2*L1–L4*L2*L2*L2–L4*
First release†††††††††††††††
Latest update
new round
maintenance only
No sourced date†
no later event
new round
†
new round
†
new version
board entry
paper revision
paper revision
no later sourced event found (as of 2026-11-16)
data
†
maintenance only
†
new version
†
data
†
new version
paper revision
†
archived
†
data
†
new round
†
new version
new round
board entry
paper revision
maintenance only
data
†
data
†
maintenance only
data
new round
data
†
paper revision
†
maintenance only
new version
†
board entry
new version
†
new round
maintenance only
†
data
maintenance only
maintenance only
results
†
board entry
paper revision
†
board entry
Status Active Active Active Round in progress Low-frequency Active Low-frequency Frozen Active Frozen Frozen Low-frequency Low-frequency Active Low-frequency Active Frozen Low-frequency Active Low-frequency Active Active Active Active Low-frequency Low-frequency Active Active Active Low-frequency Low-frequency Low-frequency Active Active Active Active Frozen Frozen Frozen Active Active Active Low-frequency Active
Wet-lab refreshNewStaticNewNewNewAccumulatingStaticStaticStaticStaticStaticStaticStaticNewNewStaticStaticAccumulatingAccumulatingStaticNewStaticStaticAccumulatingNewNewStaticAccumulatingNewStaticStaticStaticStaticStaticAccumulatingNewStaticStaticStaticStaticAccumulatingStaticStaticStatic
Models reported15842351224052200601002436110351003011901230311
Current ranking No ranking
CASP has no single overall board and CASP17 results are not out yet.
Not comparable
5 results on different protocols, no single ranking
Third-party
No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the c…
No ranking
There is no ranking this round, and the entries come from the CASP16 assessment of the joint round rather tha…
Board
Kaggle private-board rank 1 team
0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10)
Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board ·
Best site model: AlphaFold 3
Board
success_rate 15.068; rmsd_success_rate 17.808; mean_lddt_pli 0.304; coverage 96.053
success_rate: share of predictions with LDDT-PLI > 0.8 and BiSyRMSD < 2; the page also lists rmsd_success_rate, mean_lddt_pli and coverage · Runs N' Poses (Polaris preview benchmark, plinder-org/runs-n-poses)
Board ·
Board
Protein-Protein 72.93% / Antibody-Antigen 47.90% / Protein-Ligand 64.90%; Protein-RNA 62.32%, Protein-DNA 79.18%, RNA Monomer 0.61, DNA Monomer 0.53
Interface success rate (% ; DockQ>=0.23, protein-ligand requires LRMSD<2 A and LDDT-PLI>0.8); monomers use LDDT · FoldBench official repo leaderboard, full set (targets released after 2023-01)
Board ·
Not comparable
3 results on different protocols, no single ranking
Board
None of the 22 official boards dates its entries, so every score is withheld and no entry can serve as headli…
Not comparable
4 results on different protocols, no single ranking
Not comparable
5 results on different protocols, no single ranking
Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
Self-reported
Sum 18.362 (Table 3, 'Goal-directed hit generation results', 23 PMO tasks, AUC top-10)
AUC top-10, summed over 23 oracles (max 23), 10,000-call budget · PMO 23 oracles (QED, DRD2, GSK3B, JNK3 plus 19 GuacaMol-derived), oracle functions via TDC
Self-reported · stale
not independently reproduced
No ranking
The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a…
No ranking
Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET…
Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
Not comparable
1 result on different protocols, no single ranking
Board
AIDO Protein-RAG (16B)
0.518 (SE 0.0)
Average Spearman (zero-shot, DMS substitutions) · ProteinGym v1.3, 217 substitution DMS assays, zero-shot
Board · stale
Best site model: VenusREM (#2)
No ranking
CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline…
Not comparable
3 results on different protocols, no single ranking
No ranking
The current round has no results and the organiser page shows no ranks or scores.
Board
RFD4-Proteina (indexed + medium thinking)
55.06 (± 1.35)
MotifBench score (0-100) · 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline
Board ·
Best site model: RFdiffusion3 (indexed) (#2)
Board
ProteinMPNN
0.29
Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board ·
Best site model: Boltz-2 (ipSAE) (#4)
Third-party
The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is sho…
Board
AuraIDE / AuraBind (Aureka Biotechnologies, internal model)
KD 94.7 pM (KinExA); best experimental antibody 113 pM
Measured KD (HT-SPR and KinExA) plus five developability screens; placement per task · AIntibody round 1, 3 challenges, 511 designs from 29 organizations, SARS-CoV-2 spike RBD; all designs made and measured in one lab
Board ·
Board
Anonymous
0.708
Spearman correlation per property on the private held-out set · AbDev competition 2025, 80 private held-out antibodies, 5 properties
Board ·
Self-reported
The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given.
Not comparable
5 results on different protocols, no single ranking
Third-party
ClinPred / MetaRNN
AUROC 0.92
AUROC (pathogenic vs benign missense) · CAGI6 Annotate-All-Missense full evaluation set (60 predictors)
Third-party · stale
No ranking
No official board and no site model reports on these suites, so there is no headline number.
Not comparable
3 results on different protocols, no single ranking
Board
CADD
0.712 (se 0.038)
AUPRC weighted-averaged by chromosome (main metric), with standard error · TraitGym Mendelian traits (matched_9) and Complex traits (matched_9), subset 'all', zero-shot and linear-probing tracks
Board · stale
Best site model: Evo2 40B (#5)
Board
SpliceAI
0.674
Per-task metrics with no single official ranking metric: conditional log loss (bits/base, lower better), macro Spearman correlation, top-1 tissue identification accuracy · BEND 2.0 Table 1 snapshot: 8 endpoints across 5 capability categories, 17 model-interface rows
Board ·
Best site model: AlphaGenome (#2)
Board
0.2120 (Ribozyme 0.1081 / tRNA 0.4310 / Aptamer 0.0970)
Signed Spearman correlation, macro-averaged over three ncRNA categories with equal category weight (from v0.1.1) · RNAGym fitness leaderboard, ncRNA only (31 assays); mRNA coding/splicing assays excluded
Board · †
Board
The board orders by scaled overall mean but shows no numeric overall score; derived means are not used as a h…
Board
rank 1 of 300+ final-round teams (board publishes ranks only, no numeric scores)
VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics · Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations)
Board ·
No ranking
No official board and no site model reports a GuacaMol score, so no headline is given.
Not comparable
3 results on different protocols, no single ranking
Board
82.0
Accuracy (%) (macro-F1 mostly not reported), reasoning-required setting · PQA-L test (500 questions)
Board · stale
Board
0.609 (Tools + high); Base 0.345
LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' · LABBench2 (1,912 tasks, 11 categories)
Board ·
Board
mckpt2
Yes/No F1 0.9481, Acc. 0.9524; Factoid MRR 0.2000; List F1 0.1537
Yes/no macro-F1 as the ranking key of Tables 7 and 8, plus factoid MRR and list F1 · BioASQ Task 14b batch 2 exact answers, phase A+ (Table 7) and phase B (Table 8); preliminary results
Board · †
Board
0.523
Average score over the successor's 20 tasks, scored programmatically against source papers · BixBench3 (successor benchmark, 20 research-study-scale tasks)
Board ·
Self-reported
The source has no rank-1 entry: entries are unranked or protocol-specific, so the caveat is shown as for a no…
Third-party
83.70%
Final-answer accuracy, mean of three full runs, scored by a judge model · Third-party board run: Terminus 2 harness, 200 turns / 4 h per task
Third-party · †
Structure & co-folding · 7 benchmarks · board 3 · third-party 1 · not comparable 1 · no ranking 2
01 CASP A
Evidence level
L4*
First release
Latest update
new round
Status
Active
Wet-lab refresh
New
Models reported
15
Current ranking
No ranking
CASP has no single overall board and CASP17 results are not out yet.
05 PoseBusters C B
Evidence level
L3*
First release
Latest update
maintenance only
Status
Active
Wet-lab refresh
Static
Models reported
8
Current ranking
Not comparable
5 results on different protocols, no single ranking
15 CAMEO A
Evidence level
L4
First release
†
Latest update
No sourced date†
no later event
Status
Active
Wet-lab refresh
New
Models reported
4
Current ranking
Third-party
No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the c…
22 CAPRI A
Evidence level
L4*
First release
†
Latest update
new round
Status
Round in progress
Wet-lab refresh
New
Models reported
2
Current ranking
No ranking
There is no ranking this round, and the entries come from the CASP16 assessment of the joint round rather tha…
25 RNA-Puzzles A B
Evidence level
L2–L4*
First release
Latest update
†
new round
Status
Low-frequency
Wet-lab refresh
New
Models reported
3
Current ranking
Board
Kaggle private-board rank 1 team
0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10)
Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board ·
Best site model: AlphaFold 3
30 PLINDER / PINDER / Runs N' Poses C B
Evidence level
L3*
First release
†
Latest update
†
new version
Status
Active
Wet-lab refresh
Accumulating
Models reported
5
Current ranking
Board
success_rate 15.068; rmsd_success_rate 17.808; mean_lddt_pli 0.304; coverage 96.053
success_rate: share of predictions with LDDT-PLI > 0.8 and BiSyRMSD < 2; the page also lists rmsd_success_rate, mean_lddt_pli and coverage · Runs N' Poses (Polaris preview benchmark, plinder-org/runs-n-poses)
Board ·
31 FoldBench B
Evidence level
L3*
First release
Latest update
board entry
Status
Low-frequency
Wet-lab refresh
Static
Models reported
12
Current ranking
Board
Protein-Protein 72.93% / Antibody-Antigen 47.90% / Protein-Ligand 64.90%; Protein-RNA 62.32%, Protein-DNA 79.18%, RNA Monomer 0.61, DNA Monomer 0.53
Interface success rate (% ; DockQ>=0.23, protein-ligand requires LRMSD<2 A and LDDT-PLI>0.8); monomers use LDDT · FoldBench official repo leaderboard, full set (targets released after 2023-01)
Board ·
Small molecules · 10 benchmarks · board 1 · self-reported 1 · third-party 2 · not comparable 4 · no ranking 2
03 MoleculeNet (+OGB) B
Evidence level
L2–L3*
First release
Latest update
paper revision
Status
Frozen
Wet-lab refresh
Static
Models reported
2
Current ranking
Not comparable
3 results on different protocols, no single ranking
04 TDC (ADMET + DTI) B
Evidence level
L2–L3*
First release
Latest update
paper revision
Status
Active
Wet-lab refresh
Static
Models reported
4
Current ranking
Board
None of the 22 official boards dates its entries, so every score is withheld and no entry can serve as headli…
06 CASF-2016 B
Evidence level
L2*
First release
Latest update
no later sourced event found (as of 2026-11-16)
Status
Frozen
Wet-lab refresh
Static
Models reported
0
Current ranking
Not comparable
4 results on different protocols, no single ranking
07 USPTO family (+PaRoutes) B
Evidence level
L2–L3*
First release
Latest update
data
Status
Frozen
Wet-lab refresh
Static
Models reported
5
Current ranking
Not comparable
5 results on different protocols, no single ranking
12 DUD-E / LIT-PCBA B
Evidence level
L2*
First release
Latest update
†
maintenance only
Status
Low-frequency
Wet-lab refresh
Static
Models reported
2
Current ranking
Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
19 PMO (Practical Molecular Optimization) D
Evidence level
L2*
First release
Latest update
†
new version
Status
Low-frequency
Wet-lab refresh
Static
Models reported
2
Current ranking
Self-reported
Sum 18.362 (Table 3, 'Goal-directed hit generation results', 23 PMO tasks, AUC top-10)
AUC top-10, summed over 23 oracles (max 23), 10,000-call budget · PMO 23 oracles (QED, DRD2, GSK3B, JNK3 plus 19 GuacaMol-derived), oracle functions via TDC
Self-reported · stale
not independently reproduced
20 CACHE A
Evidence level
L5*
First release
†
Latest update
†
data
Status
Active
Wet-lab refresh
New
Models reported
0
Current ranking
No ranking
The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a…
27 Polaris / ASAP / OpenADMET A C
Evidence level
L4*
First release
Latest update
†
new version
Status
Low-frequency
Wet-lab refresh
New
Models reported
0
Current ranking
No ranking
Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET…
33 PoseX B
Evidence level
L3*
First release
Latest update
paper revision
Status
Active
Wet-lab refresh
Static
Models reported
6
Current ranking
Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
44 FS-Mol B
Evidence level
L3*
First release
Latest update
†
archived
Status
Frozen
Wet-lab refresh
Static
Models reported
0
Proteins · 5 benchmarks · board 2 · not comparable 1 · no ranking 2
02 ProteinGym B
Evidence level
L2–L3*
First release
†
Latest update
†
data
Status
Low-frequency
Wet-lab refresh
Accumulating
Models reported
10
Current ranking
Board
AIDO Protein-RAG (16B)
0.518 (SE 0.0)
Average Spearman (zero-shot, DMS substitutions) · ProteinGym v1.3, 217 substitution DMS assays, zero-shot
Board · stale
Best site model: VenusREM (#2)
17 CAFA A
Evidence level
L4–L5*
First release
†
Latest update
†
new round
Status
Active
Wet-lab refresh
Accumulating
Models reported
0
Current ranking
No ranking
CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline…
26 FLIP / FLIP2 B
Evidence level
L2–L3*
First release
Latest update
†
new version
Status
Low-frequency
Wet-lab refresh
Static
Models reported
2
Current ranking
Not comparable
3 results on different protocols, no single ranking
28 Adaptyv competitions / BenchBB A B
Evidence level
L5*
First release
†
Latest update
new round
Status
Active
Wet-lab refresh
New
Models reported
4
Current ranking
No ranking
The current round has no results and the organiser page shows no ranks or scores.
40 MotifBench D
Evidence level
L2*
First release
Latest update
board entry
Status
Active
Wet-lab refresh
Static
Models reported
3
Current ranking
Board
RFD4-Proteina (indexed + medium thinking)
55.06 (± 1.35)
MotifBench score (0-100) · 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline
Board ·
Best site model: RFdiffusion3 (indexed) (#2)
Antibodies · 4 benchmarks · board 3 · third-party 1
34 AbBiBench B
Evidence level
L2–L3*
First release
Latest update
paper revision
Status
Active
Wet-lab refresh
Static
Models reported
6
Current ranking
Board
ProteinMPNN
0.29
Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board ·
Best site model: Boltz-2 (ipSAE) (#4)
35 FLAb / FLAb2 B
Evidence level
L2–L3*
First release
Latest update
maintenance only
Status
Active
Wet-lab refresh
Accumulating
Models reported
1
Current ranking
Third-party
The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is sho…
36 AIntibody A
Evidence level
L5*
First release
Latest update
data
Status
Low-frequency
Wet-lab refresh
New
Models reported
1
Current ranking
Board
AuraIDE / AuraBind (Aureka Biotechnologies, internal model)
KD 94.7 pM (KinExA); best experimental antibody 113 pM
Measured KD (HT-SPR and KinExA) plus five developability screens; placement per task · AIntibody round 1, 3 challenges, 511 designs from 29 organizations, SARS-CoV-2 spike RBD; all designs made and measured in one lab
Board ·
43 Ginkgo GDPa1 A B
Evidence level
L4*
First release
†
Latest update
†
data
Status
Low-frequency
Wet-lab refresh
New
Models reported
0
Current ranking
Board
Anonymous
0.708
Spearman correlation per property on the private held-out set · AbDev competition 2025, 80 private held-out antibodies, 5 properties
Board ·
Genomics & RNA · 8 benchmarks · board 3 · self-reported 1 · third-party 1 · not comparable 2 · no ranking 1
08 Basenji/Enformer test set B
Evidence level
L2–L3*
First release
†
Latest update
†
maintenance only
Status
Active
Wet-lab refresh
Static
Models reported
3
Current ranking
Self-reported
The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given.
11 ClinVar hold-out protocol B
Evidence level
L2–L3*
First release
†
Latest update
data
Status
Active
Wet-lab refresh
Accumulating
Models reported
5
Current ranking
Not comparable
5 results on different protocols, no single ranking
13 CAGI A
Evidence level
L4–L5*
First release
†
Latest update
new round
Status
Active
Wet-lab refresh
New
Models reported
1
Current ranking
Third-party
ClinPred / MetaRNN
AUROC 0.92
AUROC (pathogenic vs benign missense) · CAGI6 Annotate-All-Missense full evaluation set (60 predictors)
Third-party · stale
16 GUE + Nucleotide Transformer benchmark B
Evidence level
L2–L3*
First release
Latest update
data
Status
Low-frequency
Wet-lab refresh
Static
Models reported
0
Current ranking
No ranking
No official board and no site model reports on these suites, so there is no headline number.
21 DREAM5 / BEELINE / CausalBench A B C
Evidence level
L2–L5*
First release
†
Latest update
†
paper revision
Status
Low-frequency
Wet-lab refresh
Static
Models reported
0
Current ranking
Not comparable
3 results on different protocols, no single ranking
37 TraitGym B
Evidence level
L3*
First release
Latest update
†
maintenance only
Status
Low-frequency
Wet-lab refresh
Static
Models reported
3
Current ranking
Board
CADD
0.712 (se 0.038)
AUPRC weighted-averaged by chromosome (main metric), with standard error · TraitGym Mendelian traits (matched_9) and Complex traits (matched_9), subset 'all', zero-shot and linear-probing tracks
Board · stale
Best site model: Evo2 40B (#5)
38 BEND B
Evidence level
L2*
First release
Latest update
new version
Status
Active
Wet-lab refresh
Static
Models reported
0
Current ranking
Board
SpliceAI
0.674
Per-task metrics with no single official ranking metric: conditional log loss (bits/base, lower better), macro Spearman correlation, top-1 tissue identification accuracy · BEND 2.0 Table 1 snapshot: 8 endpoints across 5 capability categories, 17 model-interface rows
Board ·
Best site model: AlphaGenome (#2)
39 RNAGym B
Evidence level
L2*
First release
Latest update
†
board entry
Status
Active
Wet-lab refresh
Static
Models reported
1
Current ranking
Board
0.2120 (Ribozyme 0.1081 / tRNA 0.4310 / Aptamer 0.0970)
Signed Spearman correlation, macro-averaged over three ncRNA categories with equal category weight (from v0.1.1) · RNAGym fitness leaderboard, ncRNA only (31 assays); mRNA coding/splicing assays excluded
Board · †
Single-cell & perturbation · 2 benchmarks · board 2
09 scIB (+Open Problems) B C
Evidence level
L2*
First release
Latest update
new version
Status
Active
Wet-lab refresh
Accumulating
Models reported
1
Current ranking
Board
The board orders by scaled overall mean but shows no numeric overall score; derived means are not used as a h…
24 Virtual Cell Challenge and perturbation benchmarks A B C
Evidence level
L2–L4*
First release
Latest update
†
new round
Status
Active
Wet-lab refresh
New
Models reported
9
Molecule generation · 2 benchmarks · not comparable 1 · no ranking 1
10 GuacaMol (+MOSES) D
Evidence level
L2*
First release
Latest update
maintenance only
Status
Frozen
Wet-lab refresh
Static
Models reported
0
Current ranking
No ranking
No official board and no site model reports a GuacaMol score, so no headline is given.
32 CrossDocked2020 D
Evidence level
L2*
First release
†
Latest update
†
data
Status
Frozen
Wet-lab refresh
Static
Models reported
1
Current ranking
Not comparable
3 results on different protocols, no single ranking
Text, LLMs & agents · 6 benchmarks · board 4 · self-reported 1 · third-party 1
14 PubMedQA B E
Evidence level
L1
First release
Latest update
maintenance only
Status
Frozen
Wet-lab refresh
Static
Models reported
2
Current ranking
Board
82.0
Accuracy (%) (macro-F1 mostly not reported), reasoning-required setting · PQA-L test (500 questions)
Board · stale
18 LAB-Bench / LABBench2 E
Evidence level
L1–L2*
First release
Latest update
maintenance only
Status
Active
Wet-lab refresh
Static
Models reported
3
Current ranking
Board
0.609 (Tools + high); Base 0.345
LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' · LABBench2 (1,912 tasks, 11 categories)
Board ·
23 BioASQ A B
Evidence level
L1–L4*
First release
†
Latest update
results
Status
Active
Wet-lab refresh
Accumulating
Models reported
0
Current ranking
Board
mckpt2
Yes/No F1 0.9481, Acc. 0.9524; Factoid MRR 0.2000; List F1 0.1537
Yes/no macro-F1 as the ranking key of Tables 7 and 8, plus factoid MRR and list F1 · BioASQ Task 14b batch 2 exact answers, phase A+ (Table 7) and phase B (Table 8); preliminary results
Board · †
29 BixBench E
Evidence level
L2*
First release
†
Latest update
†
board entry
Status
Active
Wet-lab refresh
Static
Models reported
3
Current ranking
Board
0.523
Average score over the successor's 20 tasks, scored programmatically against source papers · BixBench3 (successor benchmark, 20 research-study-scale tasks)
Board ·
41 BioML-bench E
Evidence level
L2*
First release
Latest update
paper revision
Status
Low-frequency
Wet-lab refresh
Static
Models reported
1
Current ranking
Self-reported
The source has no rank-1 entry: entries are unranked or protocol-specific, so the caveat is shown as for a no…
42 BioMysteryBench E
Evidence level
L2–L4*
First release
Latest update
†
board entry
Status
Active
Wet-lab refresh
Static
Models reported
1
Current ranking
Third-party
83.70%
Final-answer accuracy, mean of three full runs, scored by a judge model · Third-party board run: Terminus 2 harness, 200 turns / 4 h per task
Third-party · †
By benchmark (table view, 44 rows)
The 44 benchmarks: identity, freshness and current ranking, one row per benchmark
BenchmarkDomainTypeEvidence levelFirst releaseLatest updateStatusWet-lab refreshModels reportedCurrent ranking
01 CASPStructure & co-foldingAL4*
new round
ActiveNew15 No ranking
CASP has no single overall board and CASP17 results are not out yet.
05 PoseBustersStructure & co-foldingC BL3*
maintenance only
ActiveStatic8 Not comparable
5 results on different protocols, no single ranking
15 CAMEOStructure & co-foldingAL4†No sourced date†
no later event
ActiveNew4 Third-party
No headline number: the ranking source has no rank-1 entry (entries are unranked or protocol-specific); the c…
22 CAPRIStructure & co-foldingAL4*†
new round
Round in progressNew2 No ranking
There is no ranking this round, and the entries come from the CASP16 assessment of the joint round rather tha…
25 RNA-PuzzlesStructure & co-foldingA BL2–L4*†
new round
Low-frequencyNew3 Board
Kaggle private-board rank 1 team
0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10)
Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board ·
Best site model: AlphaFold 3
30 PLINDER / PINDER / Runs N' PosesStructure & co-foldingC BL3*††
new version
ActiveAccumulating5 Board
success_rate 15.068; rmsd_success_rate 17.808; mean_lddt_pli 0.304; coverage 96.053
success_rate: share of predictions with LDDT-PLI > 0.8 and BiSyRMSD < 2; the page also lists rmsd_success_rate, mean_lddt_pli and coverage · Runs N' Poses (Polaris preview benchmark, plinder-org/runs-n-poses)
Board ·
31 FoldBenchStructure & co-foldingBL3*
board entry
Low-frequencyStatic12 Board
Protein-Protein 72.93% / Antibody-Antigen 47.90% / Protein-Ligand 64.90%; Protein-RNA 62.32%, Protein-DNA 79.18%, RNA Monomer 0.61, DNA Monomer 0.53
Interface success rate (% ; DockQ>=0.23, protein-ligand requires LRMSD<2 A and LDDT-PLI>0.8); monomers use LDDT · FoldBench official repo leaderboard, full set (targets released after 2023-01)
Board ·
03 MoleculeNet (+OGB)Small moleculesBL2–L3*
paper revision
FrozenStatic2 Not comparable
3 results on different protocols, no single ranking
04 TDC (ADMET + DTI)Small moleculesBL2–L3*
paper revision
ActiveStatic4 Board
None of the 22 official boards dates its entries, so every score is withheld and no entry can serve as headli…
06 CASF-2016Small moleculesBL2*
no later sourced event found (as of 2026-11-16)
FrozenStatic0 Not comparable
4 results on different protocols, no single ranking
07 USPTO family (+PaRoutes)Small moleculesBL2–L3*
data
FrozenStatic5 Not comparable
5 results on different protocols, no single ranking
12 DUD-E / LIT-PCBASmall moleculesBL2*†
maintenance only
Low-frequencyStatic2 Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
19 PMO (Practical Molecular Optimization)Small moleculesDL2*†
new version
Low-frequencyStatic2 Self-reported
Sum 18.362 (Table 3, 'Goal-directed hit generation results', 23 PMO tasks, AUC top-10)
AUC top-10, summed over 23 oracles (max 23), 10,000-call budget · PMO 23 oracles (QED, DRD2, GSK3B, JNK3 plus 19 GuacaMol-derived), oracle functions via TDC
Self-reported · stale
not independently reproduced
20 CACHESmall moleculesAL5*††
data
ActiveNew0 No ranking
The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a…
27 Polaris / ASAP / OpenADMETSmall moleculesA CL4*†
new version
Low-frequencyNew0 No ranking
Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET…
33 PoseXSmall moleculesBL3*
paper revision
ActiveStatic6 Third-party
No headline number: the ranking source gives no rank-1 entry (all entries are unranked or protocol-specific);…
44 FS-MolSmall moleculesBL3*†
archived
FrozenStatic0 Not comparable
1 result on different protocols, no single ranking
02 ProteinGymProteinsBL2–L3*††
data
Low-frequencyAccumulating10 Board
AIDO Protein-RAG (16B)
0.518 (SE 0.0)
Average Spearman (zero-shot, DMS substitutions) · ProteinGym v1.3, 217 substitution DMS assays, zero-shot
Board · stale
Best site model: VenusREM (#2)
17 CAFAProteinsAL4–L5*††
new round
ActiveAccumulating0 No ranking
CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline…
26 FLIP / FLIP2ProteinsBL2–L3*†
new version
Low-frequencyStatic2 Not comparable
3 results on different protocols, no single ranking
28 Adaptyv competitions / BenchBBProteinsA BL5*†
new round
ActiveNew4 No ranking
The current round has no results and the organiser page shows no ranks or scores.
40 MotifBenchProteinsDL2*
board entry
ActiveStatic3 Board
RFD4-Proteina (indexed + medium thinking)
55.06 (± 1.35)
MotifBench score (0-100) · 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline
Board ·
Best site model: RFdiffusion3 (indexed) (#2)
34 AbBiBenchAntibodiesBL2–L3*
paper revision
ActiveStatic6 Board
ProteinMPNN
0.29
Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board ·
Best site model: Boltz-2 (ipSAE) (#4)
35 FLAb / FLAb2AntibodiesBL2–L3*
maintenance only
ActiveAccumulating1 Third-party
The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is sho…
36 AIntibodyAntibodiesAL5*
data
Low-frequencyNew1 Board
AuraIDE / AuraBind (Aureka Biotechnologies, internal model)
KD 94.7 pM (KinExA); best experimental antibody 113 pM
Measured KD (HT-SPR and KinExA) plus five developability screens; placement per task · AIntibody round 1, 3 challenges, 511 designs from 29 organizations, SARS-CoV-2 spike RBD; all designs made and measured in one lab
Board ·
43 Ginkgo GDPa1AntibodiesA BL4*††
data
Low-frequencyNew0 Board
Anonymous
0.708
Spearman correlation per property on the private held-out set · AbDev competition 2025, 80 private held-out antibodies, 5 properties
Board ·
08 Basenji/Enformer test setGenomics & RNABL2–L3*††
maintenance only
ActiveStatic3 Self-reported
The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given.
11 ClinVar hold-out protocolGenomics & RNABL2–L3*†
data
ActiveAccumulating5 Not comparable
5 results on different protocols, no single ranking
13 CAGIGenomics & RNAAL4–L5*†
new round
ActiveNew1 Third-party
ClinPred / MetaRNN
AUROC 0.92
AUROC (pathogenic vs benign missense) · CAGI6 Annotate-All-Missense full evaluation set (60 predictors)
Third-party · stale
16 GUE + Nucleotide Transformer benchmarkGenomics & RNABL2–L3*
data
Low-frequencyStatic0 No ranking
No official board and no site model reports on these suites, so there is no headline number.
21 DREAM5 / BEELINE / CausalBenchGenomics & RNAA B CL2–L5*††
paper revision
Low-frequencyStatic0 Not comparable
3 results on different protocols, no single ranking
37 TraitGymGenomics & RNABL3*†
maintenance only
Low-frequencyStatic3 Board
CADD
0.712 (se 0.038)
AUPRC weighted-averaged by chromosome (main metric), with standard error · TraitGym Mendelian traits (matched_9) and Complex traits (matched_9), subset 'all', zero-shot and linear-probing tracks
Board · stale
Best site model: Evo2 40B (#5)
38 BENDGenomics & RNABL2*
new version
ActiveStatic0 Board
SpliceAI
0.674
Per-task metrics with no single official ranking metric: conditional log loss (bits/base, lower better), macro Spearman correlation, top-1 tissue identification accuracy · BEND 2.0 Table 1 snapshot: 8 endpoints across 5 capability categories, 17 model-interface rows
Board ·
Best site model: AlphaGenome (#2)
39 RNAGymGenomics & RNABL2*†
board entry
ActiveStatic1 Board
0.2120 (Ribozyme 0.1081 / tRNA 0.4310 / Aptamer 0.0970)
Signed Spearman correlation, macro-averaged over three ncRNA categories with equal category weight (from v0.1.1) · RNAGym fitness leaderboard, ncRNA only (31 assays); mRNA coding/splicing assays excluded
Board · †
09 scIB (+Open Problems)Single-cell & perturbationB CL2*
new version
ActiveAccumulating1 Board
The board orders by scaled overall mean but shows no numeric overall score; derived means are not used as a h…
24 Virtual Cell Challenge and perturbation benchmarksSingle-cell & perturbationA B CL2–L4*†
new round
ActiveNew9 Board
rank 1 of 300+ final-round teams (board publishes ranks only, no numeric scores)
VCC 2025 final rank aggregating PDS, DES and MAE, with the Generalist Prize as mean rank over 7 metrics · Virtual Cell Challenge 2025 final test set (H1 hESC, held-out CRISPRi perturbations)
Board ·
10 GuacaMol (+MOSES)Molecule generationDL2*
maintenance only
FrozenStatic0 No ranking
No official board and no site model reports a GuacaMol score, so no headline is given.
32 CrossDocked2020Molecule generationDL2*††
data
FrozenStatic1 Not comparable
3 results on different protocols, no single ranking
14 PubMedQAText, LLMs & agentsB EL1
maintenance only
FrozenStatic2 Board
82.0
Accuracy (%) (macro-F1 mostly not reported), reasoning-required setting · PQA-L test (500 questions)
Board · stale
18 LAB-Bench / LABBench2Text, LLMs & agentsEL1–L2*
maintenance only
ActiveStatic3 Board
0.609 (Tools + high); Base 0.345
LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' · LABBench2 (1,912 tasks, 11 categories)
Board ·
23 BioASQText, LLMs & agentsA BL1–L4*†
results
ActiveAccumulating0 Board
mckpt2
Yes/No F1 0.9481, Acc. 0.9524; Factoid MRR 0.2000; List F1 0.1537
Yes/no macro-F1 as the ranking key of Tables 7 and 8, plus factoid MRR and list F1 · BioASQ Task 14b batch 2 exact answers, phase A+ (Table 7) and phase B (Table 8); preliminary results
Board · †
29 BixBenchText, LLMs & agentsEL2*††
board entry
ActiveStatic3 Board
0.523
Average score over the successor's 20 tasks, scored programmatically against source papers · BixBench3 (successor benchmark, 20 research-study-scale tasks)
Board ·
41 BioML-benchText, LLMs & agentsEL2*
paper revision
Low-frequencyStatic1 Self-reported
The source has no rank-1 entry: entries are unranked or protocol-specific, so the caveat is shown as for a no…
42 BioMysteryBenchText, LLMs & agentsEL2–L4*†
board entry
ActiveStatic1 Third-party
83.70%
Final-answer accuracy, mean of three full runs, scored by a judge model · Third-party board run: Terminus 2 harness, 200 turns / 4 h per task
Third-party · †

The ranking row shows only each benchmark's current first place; it cannot be compared across columns. Board = the organizer's leaderboard; self-reported = the developers' own report; third-party = an independent evaluation. † not fully verified (hover for the basis). * qualifier on the evidence level. Stale = snapshot more than 12 months old. Badges: board 18 · not comparable 9 · no ranking 8 · third-party 6 · self-reported 3.

44
benchmarks
112
models
131
established pairings
11
benchmarks no model reports
35
models report none of the 44
7
trained on own wet-lab data
The short version
  • Of the 44 benchmarks, 11 have no established result from any of the 112 models, and 35 of the 112 models have none on the 44. Most public evidence for the newest models is wet-lab hit rate on self-chosen targets, not a shared test.
  • Only 11 of 44 benchmarks produce fresh experimental data each round. The rest are static or grow from public records, so they cannot show whether a model works on a target nobody has measured yet.
  • Of 196 recorded pairings, 131 are established: 38 official leaderboard or results, 31 developer reports on the official protocol, 29 on a modified protocol and 33 retrospective uses of released data. 10 are only cited as comparisons and 55 could not be confirmed.
  • Only 7 of 112 models trained on wet-lab data their developers generated for the purpose. Most use public databases, and many lab-validated design models do not disclose their training data.
  • Current rankings exist for few benchmarks: 18 have an organizer board, 6 only third-party and 3 only self-reported standings; 9 are not comparable across protocols and 8 have no ranking. 0 of 44 benchmarks reach L6, independent replication.

How to read this

A model paper's best result holds only on the benchmark it chose. Change the split, metric or time window and the position can differ. Rulers also differ: some truths are measured in blind experiments, others come from scoring functions; some test sets are well separated from training data, others leak widely. This page covers 44 widely used benchmarks and 112 models released in the past year, noting for each pairing whether it is author-reported, an official leaderboard or a third-party evaluation. It records what each benchmark tests, its evidence level and whether it is still usable. Scores appear only in each benchmark's own ranking, verbatim, with metric, source type and date, and are never compared across benchmarks.

Every pairing links one model to one benchmark and has one relation type and one source type (board, self-reported, third-party, or no confirmable source). Only established relations are counted as “reported”.

Relation typeMeaningPairingsCounted
Official leaderboard or resultsorganizer results or leaderboard under an identifiable name (counted, established)38yes
Developer report, official protocoldeveloper reports on the official split/protocol, not on a board (counted, established)31yes
Developer report, modified protocoldeveloper reports with different filtering/subset/sampling, or self-reported participation the organizer has not confirmed (counted, established)29yes
Third-party evaluationonly third parties ran the model on the benchmark (ADDED this round by owner decision; counted, established)0yes
Retrospective dataset usepost-hoc reuse of a released round's data (established under this round's definition, but shown separately and never as participation)33yes
Cited as comparison onlythe model appears only as a baseline number copied from elsewhere, or only the tool is used (NOT established)10no
Unconfirmedclaimed or second-hand, not verifiable (NOT established)55no

Evidence levels

Levels describe the type of evidence a benchmark gives, not how good it is. A benchmark can span several, and scores on different levels answer different questions: a 0.9 correlation on L2 and a 10% hit rate on L5 are not comparable.

LevelMeaning
L1Knowledge recall (question answering, multiple choice)
L2Retrospective random or in-distribution split
L3Stratified or out-of-distribution split (similarity, scaffold, time)
L4Blind test with hidden truth (truth does not exist at submission)
L5Prospective wet-lab experiment
L6Independent replication

Benchmark types

TypeMeaningExamples
APeriodic blind challenge; new questions and new truth each roundCASP, CAPRI, CAGI, CACHE, AIntibody
BStatic dataset suite; download and runProteinGym, MoleculeNet, TDC, FoldBench
CInfrastructure: validators, similarity-split tools, evaluation protocolsPoseBusters checker, PLINDER/PINDER splits
DComputational-oracle benchmark; truth comes from a scoring function or fold read-backGuacaMol, PMO, CrossDocked2020, MotifBench
ETask environments for agents and LLMsLAB-Bench, BixBench, BioML-bench, BioMysteryBench

Ten findings

  1. Only a few blind tests can show methodological progress12 of 44 benchmarks reach L4 (blind tests with hidden truth) and 6 reach L5 (prospective wet-lab experiments). CASP, CAPRI, CAFA, CAGI, CAMEO, CACHE and AIntibody are the long-running or recurring ones. For the rest, isolation and generalization controls are weaker.
  2. The most influential benchmarks are often the weakestMoleculeNet, USPTO, CASF-2016, DUD-E and GuacaMol owe part of their results to memorization and constructed shortcuts. Static sets that can be tuned indefinitely are the ones that accumulate influence.
  3. Simple baselines repeatedly match deep modelsRetrieval on CASF-2016, XGBoost on a TDC task, ECFP against pretrained embeddings, PrediXcan against Enformer, additive or linear baselines in perturbation, randomly initialized DNA models, template retrieval in RNA 3D, and alignment-based methods on TraitGym.
  4. Metric validity is a field-wide weaknessMost benchmarks report no confidence intervals, and gaps among leading entries are often within sampling noise. PubMedQA, PoseBusters V2 and several FoldBench tasks have small test sets; FoldBench's authors advise against interpreting small gaps.
  5. Leakage now means similarity must be first-classMany CASF-2016 test complexes have near-duplicates in training, and most USPTO-50K test reactions match training templates. Under PLINDER and FoldBench, co-folding models degrade in low-similarity bins, so later-cutoff models should report later subsets.
  6. Antibody evaluation is new, and early findings are negativeAll 4 antibody benchmarks first appeared in 2024 or later. Results do not transfer across tasks, cross-validation exceeds held-out performance, zero-shot correlation is often non-significant, and specialized fine-tuning is not reliably helpful. Most antibody-specific models report only on self-built sets.
  7. Newer benchmarks are almost all staticOf the 14 added entries, 12 produce no new experimental data each round; CrossDocked2020 and MotifBench use docking scores and fold read-back as truth. Only FoldBench is widely adopted: maintainer-run leaderboard, antibody-antigen coverage, no waiting.
  8. Official leaderboards are mostly run by maintainers52% of official-leaderboard entries sit on ProteinGym, FoldBench and AbBiBench, all run by the benchmark owners. Genuine submission-based boards are few, such as MotifBench and LABBench2/BixBench3.
  9. Agent and LLM boards exist, but version control is a new problemLABBench2 and BixBench3 have dated official leaderboards, an improvement on self-reporting. BixBench3 differs from BixBench; vendor system cards restate sub-items inconsistently; BioMysteryBench is vendor-authored with a non-public full set; PubMedQA is saturated.
  10. Infrastructure is the most underrated contributionPoseBusters (validity checks beyond RMSD), PLINDER (similarity-stratified splits), PMO (fixed oracle budget), Polaris (bootstrap and multiple-comparison statistics) and MotifBench (generate-and-refold pipeline). No benchmark provides quantitative evidence that better rankings raise wet-lab hit rates.

Who reports what

The matrix of 112 models against 44 benchmarks is mostly empty. Figure 1 shows how thin the coverage is: only 5 benchmarks are reported by eight or more models (CASP 15, FoldBench 12, ProteinGym 10, Virtual Cell Challenge and perturbation benchmarks 9, PoseBusters 8), and 11 by none. Figure 2 shows the other side, how few benchmarks a typical model reports. Figure 3 asks who vouches for the numbers.

Figure 1. Which benchmarks do models actually report?

Models with an established result on each of the 44 benchmarks

Core 30Supplementary 14
Counted relations: official leaderboard or results, developer reports on the official or a modified protocol, and retrospective use of released data. Pairings that are only cited as comparisons or are unconfirmed are not counted (65 of 196).
View as table
BenchmarkSetModels countedBy relation type
01 CASPCore15board 3, retrospective 12, unconfirmed 1
31 FoldBenchSupplementary12board 7, self 3, modified protocol 2
02 ProteinGymCore10board 7, self 2, retrospective 1
24 Virtual Cell Challenge and perturbation benchmarksCore9board 1, modified protocol 6, retrospective 2
05 PoseBustersCore8self 4, modified protocol 4, cited only 3
33 PoseXSupplementary6retrospective 6
34 AbBiBenchSupplementary6board 6
07 USPTO family (+PaRoutes)Core5self 4, modified protocol 1
11 ClinVar hold-out protocolCore5modified protocol 5
30 PLINDER / PINDER / Runs N' PosesCore5self 3, modified protocol 2, cited only 4
04 TDC (ADMET + DTI)Core4board 3, self 1, unconfirmed 2
15 CAMEOCore4board 1, retrospective 3, unconfirmed 1
28 Adaptyv competitions / BenchBBCore4board 2, modified protocol 1, retrospective 1
08 Basenji/Enformer test setCore3self 3
18 LAB-Bench / LABBench2Core3board 1, modified protocol 2, unconfirmed 5
25 RNA-PuzzlesCore3modified protocol 2, retrospective 1, cited only 1
29 BixBenchCore3board 1, self 1, modified protocol 1, unconfirmed 6
37 TraitGymSupplementary3modified protocol 1, retrospective 2
40 MotifBenchSupplementary3board 3
03 MoleculeNet (+OGB)Core2self 1, modified protocol 1, unconfirmed 1
12 DUD-E / LIT-PCBACore2retrospective 2, unconfirmed 3
14 PubMedQACore2self 2, unconfirmed 6
19 PMO (Practical Molecular Optimization)Core2self 1, modified protocol 1, cited only 1
22 CAPRICore2board 2, unconfirmed 1
26 FLIP / FLIP2Core2self 2
09 scIB (+Open Problems)Core1self 1, unconfirmed 2
13 CAGICore1retrospective 1
32 CrossDocked2020Supplementary1self 1
35 FLAb / FLAb2Supplementary1retrospective 1
36 AIntibodySupplementary1board 1, unconfirmed 1
39 RNAGymSupplementary1self 1, unconfirmed 1
41 BioML-benchSupplementary1retrospective 1, unconfirmed 6
42 BioMysteryBenchSupplementary1self 1, unconfirmed 6
06 CASF-2016Core0none
10 GuacaMol (+MOSES)Core0none
16 GUE + Nucleotide Transformer benchmarkCore0none
17 CAFACore0unconfirmed 1
20 CACHECore0none
21 DREAM5 / BEELINE / CausalBenchCore0unconfirmed 2
23 BioASQCore0unconfirmed 8
27 Polaris / ASAP / OpenADMETCore0cited only 1, unconfirmed 2
38 BENDSupplementary0none
43 Ginkgo GDPa1Supplementary0none
44 FS-MolSupplementary0none

Figure 2. Most models report on zero or one benchmark

Number of the 112 models, by how many of the 44 benchmarks they have an established result on

01020304050350481152734435 or morebenchmarks with an established result per model
35 models have none. They are mostly protein-design, antibody and agent systems whose public evidence is wet-lab hit rate on self-chosen targets, not a shared test.
View as table
Benchmarks per modelModels
035
148
215
37
44
5 or more3

Figure 3. Who vouches for the number?

All 196 recorded pairings by source type, grouped by model area

Structure & co-folding162416662Small molecules135627Proteins571527Antibodies1Genomics & RNA1014Single-cell & perturbation7414Molecule generation1Text, LLMs & agents83850
Self-reportedThird-partyOfficial leaderboardUnconfirmed source
Overall: 60 self-reported, 43 third-party, 38 official-leaderboard and 55 with no confirmable primary source. 52% of the official-leaderboard entries sit on three benchmarks (ProteinGym, FoldBench and AbBiBench), and on those the maintainers ran the models themselves.
View as table
Model areaSelf-reportedThird-partyOfficial leaderboardUnconfirmed source
Structure & co-folding1624166
Small molecules13536
Proteins57150
Antibodies0010
Genomics & RNA10301
Single-cell & perturbation7214
Molecule generation1000
Text, LLMs & agents82238

The full model-by-benchmark matrix, with hover details for each of the 196 recorded pairings, is on the explorer page.

  • 19 of 44 benchmarks are reported by at most one model, and 11 by none: CASF-2016, GuacaMol, GUE + Nucleotide Transformer benchmark, CAFA, CACHE, DREAM5 / BEELINE / CausalBench, BioASQ, Polaris / ASAP / OpenADMET, BEND, Ginkgo GDPa1, FS-Mol. Older ones are seen as undiscriminating or costly to enter; the newest have not yet been adopted.
  • Among the newer 14 entries, only FoldBench is genuinely adopted: maintainer-run leaderboard, antibody-antigen interfaces, no waiting for a window. All AbBiBench entries were registered by its own maintainers; most others have one to three entries.
  • Most models report on zero or one benchmark. Those with zero are mostly protein-design and agent systems whose public results are wet-lab hit rates on self-selected targets or internal metrics, with no shared yardstick.
  • Public evaluation coverage is inversely related to commercialization. Structure and co-folding have several shared yardsticks and a full matrix; protein design and antibody models, whose unit of result is targets and experimental batches, leave it sparse.
  • Unconfirmed pairings are concentrated among agent and LLM systems, whose official materials do not report these benchmarks although external lists and media fill them in. Other cases are disputes over provenance or anonymized official entries.
  • Official leaderboards are a minority and unevenly distributed. ProteinGym, FoldBench and AbBiBench account for 52% of them, all run by maintainers. Most leaderboard results therefore reflect maintainer willingness to evaluate, not industry submission habits.

How strong is the evidence?

A benchmark is only as informative as the question it can answer. Figure 4 counts the benchmarks that reach each evidence level; Figure 6 shows when they appeared and whether anyone still maintains them.

Figure 4. Evidence level: none reaches independent replication

Benchmarks that touch each evidence level (a benchmark can span several levels)

010203L127L221L312L46L50L6evidence level touched
Evidence levels describe the type of evidence, not quality: L2 is a retrospective random split, L4 a blind test with hidden truth, L5 a prospective wet-lab experiment, L6 an independent replication. 0 of 44 benchmarks reach L6.
View as table
LevelMeaningBenchmarks touching it
L1Knowledge recall (question answering, multiple choice)3
L2Retrospective random or in-distribution split27
L3Stratified or out-of-distribution split (similarity, scaffold, time)21
L4Blind test with hidden truth (truth does not exist at submission)12
L5Prospective wet-lab experiment6
L6Independent replication0

Figure 6. Many benchmarks are young, and some are no longer maintained

Each dot is one benchmark, placed by the year of its first sourced release

1995200020052010201520202025year of first release
Active or round in progressLow-frequency updatesFrozen
24 of 44 are active (1 of them with a round in progress), 13 update rarely and 7 are frozen. 22 benchmarks were first released in 2022 or later. Dates follow the stored precision; † marks a date that is not fully verified.
View as table
First releaseBenchmarkStatus
199401 CASPActive
2001 (partly verified)22 CAPRIRound in progress
2009-04-0906 CASF-2016Frozen
2010 (partly verified)13 CAGIActive
2010-09 (partly verified)17 CAFAActive
2010 (partly verified)21 DREAM5 / BEELINE / CausalBenchLow-frequency
2012-07-0512 DUD-E / LIT-PCBALow-frequency
2012 (partly verified)15 CAMEOActive
2012-02-2325 RNA-PuzzlesLow-frequency
2013-04 (partly verified)11 ClinVar hold-out protocolActive
2013 (partly verified)23 BioASQActive
2014-0207 USPTO family (+PaRoutes)Frozen
2017-03-0203 MoleculeNet (+OGB)Frozen
2017-07-10 (partly verified)08 Basenji/Enformer test setActive
2018-11-2210 GuacaMol (+MOSES)Frozen
2019-09-1314 PubMedQAFrozen
2020-11-1304 TDC (ADMET + DTI)Active
2020-05-2309 scIB (+Open Problems)Active
2020-02-20 (partly verified)32 CrossDocked2020Frozen
2021-12-01 (partly verified)20 CACHEActive
2021-11-1126 FLIP / FLIP2Low-frequency
2021-1244 FS-MolFrozen
2022-05-27 (partly verified)02 ProteinGymLow-frequency
2022-06-1016 GUE + Nucleotide Transformer benchmarkLow-frequency
2022-06-2219 PMO (Practical Molecular Optimization)Low-frequency
2023-08-1005 PoseBustersActive
2023-11-2327 Polaris / ASAP / OpenADMETLow-frequency
2023-11-2138 BENDActive
2024-07-1418 LAB-Bench / LABBench2Active
2024-08-2024 Virtual Cell Challenge and perturbation benchmarksActive
2024-07-18 (unverified)28 Adaptyv competitions / BenchBBActive
2024-07-17 (partly verified)30 PLINDER / PINDER / Runs N' PosesActive
2024-01-1535 FLAb / FLAb2Active
2024-11-0436 AIntibodyLow-frequency
2025-02-28 (partly verified)29 BixBenchActive
2025-05-2731 FoldBenchLow-frequency
2025-05-0333 PoseXActive
2025-05-2334 AbBiBenchActive
2025-02-1237 TraitGymLow-frequency
2025-06-1739 RNAGymActive
2025-02-1840 MotifBenchActive
2025-09-0441 BioML-benchLow-frequency
2025-09-08 (partly verified)43 Ginkgo GDPa1Low-frequency
2026-04-2942 BioMysteryBenchActive

Where new wet-lab data come from

A benchmark that never receives new experimental data can only tell you how a model does on what is already known. Figure 5 sorts the 44 benchmarks by how they refresh. Among the 14 supplementary entries, only 2 generate fresh experiments.

Figure 5. Where do new wet-lab data come from?

Benchmarks by how they get new data, grouped by area

Structure & co-folding4127Small molecules2810Proteins1225Antibodies2114Genomics & RNA1168Single-cell & perturbation112Molecule generation22Text, LLMs & agents156
Fresh experiments each roundGrows from public sourcesStatic data
Only 11 of 44 benchmarks generate fresh experimental data each round; 7 grow from public sources (historical annotations, PDB releases, newly written questions); 26 are fixed.
View as table
AreaFresh experimentsGrows from public sourcesStatic
Structure & co-folding412
Small molecules208
Proteins122
Antibodies211
Genomics & RNA116
Single-cell & perturbation110
Molecule generation002
Text, LLMs & agents015

A. Periodically generate new experimental data (blind or prospective)

Organizers or partners produce new measurements each round: unpublished structures, functional experiments, compounds synthesized and assayed, new Perturb-seq, or designs tested in a single lab. Some are one-off or non-periodic.

Examples: CASP, CAMEO, CAPRI, RNA-Puzzles, CAGI, CACHE, Polaris/OpenADMET, Adaptyv/BenchBB, AIntibody, GDPa1, Virtual Cell Challenge

B. Updated periodically, but not new experiments

New data come from time-delayed accumulation of historical annotations, clinical submissions, weekly PDB updates, newly written expert questions, new question versions on existing sequencing data, or community-contributed published assays.

Examples: CAFA, ClinVar protocol, PLINDER/PINDER, BioASQ, LABBench2, BioMysteryBench, FLAb/FLAb2

C. Static (reused historical public data or computational oracles)

No new data are added. ProteinGym occasionally incorporates new DMS data; AbBiBench ran wet-lab work once.

Examples: MoleculeNet, TDC, PoseBusters, CASF, USPTO, Enformer, scIB, GuacaMol, DUD-E/LIT-PCBA, PubMedQA, GUE/NT, PMO, DREAM5/BEELINE, FLIP, BixBench, FoldBench, CrossDocked2020, PoseX, TraitGym, BEND, RNAGym, MotifBench, BioML-bench, FS-Mol

Do you need your own wet-lab data to evaluate a new model?

  • Static (C) benchmarks need no new wet lab and can be run by download. The cost is retrospective conclusions only, and several leak (CASF, USPTO, DUD-E); the later a model's training cutoff, the worse the leakage.
  • Blind (A) benchmarks (CASP, CAPRI, CAMEO, CAGI, CACHE, Polaris, VCC) need no self-generated data but require waiting for the next prediction window: weekly for CAMEO, every two years for CASP. Released rounds can be run as static sets, but are no longer blind for models with later training cutoffs.
  • Own wet-lab work is needed for protein design (for example Adaptyv-style per-sequence testing, a few dozen to a few hundred sequences per round), for molecule generation and optimization (GuacaMol and PMO have only computational oracles, so real activity requires synthesis and assay), and when conclusions must generalize to own targets or chemical space (true for TDC and Polaris).

What the models are trained on

Every model page records four kinds of training data: public databases, private or licensed data, wet-lab data the developers generated specifically for training, and wet-lab work used only to validate outputs. Figure 7 assigns each model to the first group that applies.

Figure 7. What are the models trained on?

Models by training-data provenance (each model counted once, in the highest-effort group that applies)

Structure & co-folding174224Small molecules142421Proteins914226Antibodies67Genomics & RNA729Single-cell & perturbation72212Molecule generation1Text, LLMs & agents44212
Public data onlyPublic data + wet lab for validation onlyPrivate or licensed dataOwn wet-lab training dataNot disclosed
Of 112 models, only 7 trained on wet-lab data their developers generated for the purpose; 7 use private or licensed data, 34 only validate outputs in the lab, 59 use public data and 5 do not disclose it.
View as table
Model areaPublic data onlyPublic data + wet lab for validation onlyPrivate or licensed dataOwn wet-lab training dataNot disclosed
Structure & co-folding174102
Small molecules142410
Proteins914120
Antibodies06010
Genomics & RNA72000
Single-cell & perturbation72021
Molecule generation10000
Text, LLMs & agents44112
  • Only 7 of 112 models have self-generated wet-lab training data: Enchant v1/v2, EVOLVEpro, HCAbLM, ProGen3, Tahoe-x1, VERID, X-Cell. Origin-1 and Dyno Psi-1 are described by their companies as using experimental data, but this is unconfirmed in papers or released weights.
  • Using third-party wet-lab data is not self-generation. STATE, Stack, Lingshu-Cell and xTrimoSCPerturb/scFoundation use perturbation data from Tahoe, Parse or Arc VCC and do not produce the data themselves.
  • 7 of 112 models record private or licensed training data: Beacon (Inductive Bio), Chai-3, Chemformer, KERMT, MedGemma / Med-Gemini, OpenFold3, Uni-Mol3.
  • Wet lab used only to validate outputs is recorded for 37 models (34 of them in the validation-only group of Figure 7; the rest also have private or own wet-lab data), most in protein design. Apart from ProGen3, HCAbLM and VERID, none claim the results were fed back into training.
  • 53 of 112 models document public data only; 33 more use public data and do not disclose whether they also use private data. Many public sources already contain distilled or synthetic data, and 5 models disclose no training data at all.
  • Of the 7 antibody models, HCAbLM (a transgenic-mouse immune repertoire) is the only one with self-generated training data; 7 report wet-lab validation.
  • Newer models add computationally self-generated data (MD trajectories, generated reactions), which is not wet lab, and synthetic interface data now common in protein design. VERID is a new form: agent-loop online feedback.
  • LLM and agent models are not trained on wet-lab data and have sparse validation. Several have none; Biomni and C2S-Scale have very small validations, and Co-Scientist and Robin were validated by partner labs.

Choosing a benchmark set

No single benchmark answers a research question. Each scenario below pairs two to four complementary benchmarks, says what each tests and why it was chosen, which variables to control, and what the set can and cannot support.

S1 · Protein monomer and complex structure
Benchmark and what it testsWhy it is in the setVariables to control
CASP
Prospective structure prediction by track
Blind: truth does not exist at submission.Training cutoff vs target release date; template/MSA use; top-1 vs top-k; manual intervention.
CAMEO
Rolling weekly structure prediction
Shows stability rather than peak performance.Evaluation window; training cutoff.
CAPRI
Protein complex interface quality
Source of the DockQ criterion.Sampling count; manual intervention.
FoldBench
Monomer and interface tasks for cross-model comparison
Maintainer-run leaderboard; no waiting for a window.Small RNA/DNA target counts; training cutoff.

The set can support: Structure generalization within a stated round.

It cannot support: Not functional conclusions or drug-design usability.

S2 · Co-folding and ligand pose
Benchmark and what it testsWhy it is in the setVariables to control
PoseBusters V2
Geometric and physical validity of ligand poses
The 18-check validator is its main legacy.Version (V1/V2, SD/CD); training cutoff; pose-selection rule.
Runs N' Poses (PLINDER)
Docking and co-folding by similarity bin
Makes similarity stratification first-class.Report nearest-neighbour training-pose copy and physics docking baselines.
PoseX
Self-docking and cross-docking scored separately
Compares physics, AI docking and co-folding under one protocol.Report chirality and clash errors separately.
FoldBench protein-ligand subset
Protein-ligand structure
Independent check of similarity effects.Training cutoff; later-cutoff models need late-subset reporting.

The set can support: Pose accuracy by similarity bin; PLINDER-PL50 when training own models.

It cannot support: Not affinity conclusions.

S3 · Affinity scoring and virtual screening
Benchmark and what it testsWhy it is in the setVariables to control
CASF-2016
Historical affinity scoring
Alignment with the literature only.Similarity-retrieval baseline; leakage-cleaned re-tests (LP-PDBBind, CleanSplit).
LIT-PCBA
Virtual screening after de-duplication
Less leakage-prone than DUD-E.Molecule counting unit after de-duplication.
CACHE or Polaris blinds
Prospective hit finding / blind prediction
Truth generated after submission.Library-level overlap of training and evaluation sets (ChEMBL, BindingDB).

The set can support: Relative gain after leakage removal and qualitative prospective-hit evidence.

It cannot support: Leaderboard position is not evidence of screening utility.

S4 · Variant effects and protein engineering
Benchmark and what it testsWhy it is in the setVariables to control
ProteinGym
Zero-shot fitness and variant-effect ranking
Community default for protein language models.Stratify by assay type; report multiple metrics; separate leaderboard from self-chosen DMS subsets.
FLIP/FLIP2
Extrapolation splits
Tests ability along defined extrapolation axes.Training recipe and random seed (some landscapes vary widely).
CAGI MAVE challenges
Blind variant-effect prediction
Truth unpublished at submission.MSA depth and taxon; viral proteins reported separately.

The set can support: Within-assay ranking and ability along stated extrapolation axes.

It cannot support: Not a claim of best engineering model.

S5 · Variant pathogenicity
Benchmark and what it testsWhy it is in the setVariables to control
ClinVar
Time- or gene-held-out pathogenicity classification
Retrospective clinical labels.Label circularity with training features; gene-level label imbalance.
ProteinGym clinical track
Clinical variant effect
Common reference point.Separate zero-shot from supervised probes (conclusions differ for genomic models such as Evo 2).
CAGI clinical challenges
Blind clinical prediction
Only source of blind evidence here.Challenge-specific protocol.

The set can support: Retrospective classification ability; blind evidence only from CAGI.

It cannot support: Retrospective labels cannot stand in for prospective validation.

S6 · Protein binder and de novo design
Benchmark and what it testsWhy it is in the setVariables to control
Adaptyv/BenchBB
Binder design rounds measured in one lab
Wet-lab truth on a shared platform.Hit definition; designs per target disclosed; target-difficulty strata.
Own or paid prospective SPR/BLI
Binding of own designs
Answers whether designs work.Do not screen on interface-confidence metrics alone.
MotifBench
Motif scaffolding
Unified generate-and-refold pipeline.Truth is folding read-back, not experiment.

The set can support: Qualitative comparison of hit rates on specified targets.

It cannot support: A single round cannot be used to rank methods.

S7 · Antibody design and developability
Benchmark and what it testsWhy it is in the setVariables to control
AIntibody
Prospective antibody design blind
Only prospective blind in this area; next round pending.Same-lab measurements; one round per edition.
GDPa1
Held-out developability prediction
Held-out blind exposes cross-validation optimism.Report cross-validation and held-out results side by side.
AbBiBench
Zero-shot score vs measured binding
Compares structure-conditioned and antibody-specific models.Do not mix with self-defined antibody-antigen subsets.
FLAb2
Seven property classes
Zero-shot correlation across assays.Germline similarity as covariate.
FoldBench antibody-antigen
Interface structure
Structure only.Keep structure accuracy separate from functional prediction.

The set can support: Strength of zero-shot correlation with measurements; developability error on held-out sets.

It cannot support: Not that antibody design is ready for use.

S8 · Protein function annotation
Benchmark and what it testsWhy it is in the setVariables to control
CAFA
Time-delayed GO annotation, no-knowledge subset
Prospective, time-split.Report Naive and BLAST baselines; GO version and evidence filters; per sub-ontology.
Own time-split set
Temporal generalization
Self-built check.A self-chosen GO test set is not a CAFA blind; separate text-mining from sequence-only methods.

The set can support: Temporal generalization.

It cannot support: Unannotated does not mean wrong.

S9 · ADMET and few-shot new assays
Benchmark and what it testsWhy it is in the setVariables to control
TDC-ADMET
ADMET property prediction
Common reference; use multiple seeds.Strong baselines such as XGBoost with ECFP; hold out by lab source and time.
Polaris/OpenADMET blinds
Blind ADMET prediction
Unpublished real project data.Winning entries are often anonymized and cannot be claimed.
FS-Mol
Few-label adaptation with support-set budget curves
Low-label new-assay setting.Target-family isolation; binarization threshold stated.
Own time-forward backtest
Project-specific error
Reflects the actual chemical series.MoleculeNet for historical alignment only.

The set can support: Practical error on a specified chemical series; adaptation efficiency under few labels.

It cannot support: Leaderboard position is not a basis for R&D decisions.

S10 · Molecule generation and optimization
Benchmark and what it testsWhy it is in the setVariables to control
PMO
Sample-efficient optimization under a fixed oracle budget
Budget curves reveal search efficiency.Oracle call count, total compute, multiple seeds; report unconstrained baselines.
GuacaMol
Distribution and goal-directed tasks
Smoke test only.Most PMO oracles come from GuacaMol, so they are not independent evidence.
Chemistry-quality and synthesizability filters
Quality of generated molecules
Guards against shortcut scores.Apply before reporting scores.

The set can support: Search efficiency within a budget.

It cannot support: Not real activity.

S11 · Structure-conditioned molecule generation
Benchmark and what it testsWhy it is in the setVariables to control
CrossDocked2020
Pocket-conditioned generation
Standard split.State the split (original dataset and generation subset differ).
PoseBusters
Validity and interaction recovery
Physical plausibility.Report drug-likeness and synthesizability, not only docking score.
Own pockets
Docking and synthesis assessment
Tests the intended target.Report pocket similarity to training set; docking score is an oracle, not truth.

The set can support: Geometry and docking behaviour of generated molecules in given pockets.

It cannot support: Not activity or druggability.

S12 · Retrosynthesis and reaction prediction
Benchmark and what it testsWhy it is in the setVariables to control
USPTO-50K/MIT
Patent-text reaction prediction
Alignment only.Same-patent reactions can cross splits; template-retrieval baseline.
PaRoutes
Route-level evaluation
Beyond single steps.Round-trip metrics depend on a forward model and carry proxy bias.

The set can support: Reproduction at the patent-text level.

It cannot support: Whether something can be made needs synthesis validation.

S13 · DNA and regulatory genomics
Benchmark and what it testsWhy it is in the setVariables to control
GUE/NT (revised)
DNA language model tasks
Smoke test with multiple seeds.Composition-matched negatives; chromosome hold-out; small CNN baseline.
BEND
Functional tasks
Broader task set.Do not mix its two versions.
Enformer/Borzoi
Expression prediction
Report across-gene and across-individual separately.PrediXcan-type and alignment-based baselines.
TraitGym
Causal non-coding variants
Matched controls.Using its variant set is not the same as being on its leaderboard.

The set can support: Representation quality, reference-genome fit and causal-variant identification under matched controls.

It cannot support: Individual-level effects need separate validation.

S14 · RNA
Benchmark and what it testsWhy it is in the setVariables to control
RNA-Puzzles, CASP RNA, Kaggle rounds
RNA 3D blind prediction
Only blind tradition for RNA.Template stratification; family leakage.
RNAGym
Fitness and structure tasks
Broader RNA tasks.State the source of scale figures.
Secondary-structure sets
2D structure
Split by family.A self-built set containing some RNA-Puzzles targets is not an RNA-Puzzles result.

The set can support: 3D versus 2D and structure versus fitness conclusions, each independent.

It cannot support: They cannot substitute for one another.

S15 · Single-cell integration, perturbation and regulatory networks
Benchmark and what it testsWhy it is in the setVariables to control
scIB
Batch integration
Run zero-shot and fitted settings.Baselines: HVG plus PCA, scVI.
PerturBench
Covariate transfer in perturbation prediction
Tests transfer.Resample at donor and batch level.
VCC
Perturbation prediction, multiple metrics
Prospective annual challenge.Mean, no-change, additive and linear baselines; separate post-hoc evaluation from entries.
DREAM5/BEELINE
Network inference
Per-network comparison.Report per network, do not aggregate; unobserved edges are not negatives.

The set can support: Gain relative to this baseline set.

It cannot support: Not that a virtual cell is usable.

S16 · Literature assistants, bioinformatics and research agents
Benchmark and what it testsWhy it is in the setVariables to control
BioASQ
Literature question answering
Use the current batch.Version and date.
LABBench2
Lab-style tasks, with and without tools
Official leaderboard with update date.Tool access; token and time budgets; grader model.
BixBench
Bioinformatics analysis, full set and subsets
Report both.Original and successor question banks must not be listed together.
BioML-bench
End-to-end ML pipelines
Pipeline-level test.Human baseline as lower or upper bound.
BioMysteryBench
Mystery-style analysis tasks
Questions written by a model vendor; full set not public.Separate vendor-run from third-party reruns.

The set can support: Trend in gains from tools.

It cannot support: Not the absolute level of scientific ability; cite the official leaderboard update date.

Entity pages

Every benchmark and every model has its own page built from one record with the same fields: dates with their basis and verification, relation types for each pairing, the current ranking, training data in four classes, wet-lab validation with numerators and denominators, sources and corrections.

Data, method and limits

Compiled from public papers, official repositories and leaderboards, and checked in multiple independent verification rounds. No benchmark was run for this survey, and no self-measured results appear. Conflicting claims are marked unconfirmed; items not found in public sources are marked N/A (not public). Vendor press releases, single non-peer-reviewed preprint critiques and third-party aggregator figures are down-weighted and used only as orders of magnitude. Scores are shown only inside each benchmark's ranking, copied verbatim with metric, eval set, source type and date; values that could not be verified are shown as unverified and not as numbers. Results not yet published for 2026 events (CASP17, CAFA6, VCC 2026, the next AIntibody) are excluded. Snapshot date: 2026-11-16.

  • Pairings come from one relation file (196 rows). Benchmark pages, model pages, the overview table, the figures and the explorer are all generated from it, so their counts agree.
  • Scores appear only in rankings, copied verbatim with metric, eval set, source type and date. Not-comparable benchmarks show no headline score; values that could not be verified, or that lack a source link or date, are withheld.
  • Company press releases and single non-peer-reviewed preprints are treated as weak evidence and flagged where they occur. † marks a date or value that is not fully verified.
  • Fast-moving items (CASP17 results, CAFA6, Virtual Cell Challenge 2026, the next AIntibody round) were not published at the snapshot date and are excluded.

Compiled from public papers, official repositories and leaderboards; every entity page lists its sources. No benchmark was run for this survey. Snapshot 16 Nov 2026.