AI for Bio · Benchmarks

AI for Bio benchmarks: all 44 benchmarks

All 44 benchmarks in the survey, grouped by domain. Each name opens its entity page; the overview table shows them side by side.

Structure & co-folding (7)

#BenchmarkTypeEvidenceStatusModelsRanking
01CASP
Biennial blind test of protein structure prediction; remaining gains lie in complexes, RNA, ligands and ensembles.
AL4Active15 No ranking
05PoseBusters
Protein-ligand pose benchmark: an RMSD threshold plus 18 physical and chemical validity checks.
C+BL3Active8 Not comparable
15CAMEO
Weekly rolling blind test of structure-prediction servers on PDB pre-release sequences.
AL4Active4 Third-party
22CAPRI
The oldest prospective blind test for protein complexes; its classes and DockQ are the common protocol
AL4Round in progress2 No ranking
25RNA-Puzzles
Community blind test of RNA 3D structure; accuracy tracks template availability more than model type.
A+BL2–L4Low-frequency3 Board
30PLINDER / PINDER / Runs N' Poses
Docking and co-folding infrastructure that makes training-set similarity a first-class split criterion.
C+BL3Active5 Board
31FoldBench
All-atom co-folding evaluation on low-homology PDB assemblies; the antibody-antigen task discriminates most.
BL3Low-frequency12 Board

Small molecules (10)

#BenchmarkTypeEvidenceStatusModelsRanking
03MoleculeNet (+OGB)
Historical default suite for molecular representation learning: 17 public datasets, for literature alignment, not drug-discovery claims.
BL2–L3Frozen2 Not comparable
04TDC (ADMET + DTI)
De facto ADMET leaderboard; useful for method screening, not a basis for development decisions.
BL2–L3Active4 Board
06CASF-2016
Decade-old default test of protein-ligand scoring functions across four powers; frozen and heavily overlapping with training data.
BL2Frozen0 Not comparable
07USPTO family (+PaRoutes)
Default data for retrosynthesis and reaction prediction; measures reproducing patent records, not lab feasibility.
BL2–L3Frozen5 Not comparable
12DUD-E / LIT-PCBA
Two generations of virtual-screening enrichment sets: constructed decoys (DUD-E) and PubChem inactives (LIT-PCBA); both have known bias or leakage.
BL2Low-frequency2 Third-party
19PMO (Practical Molecular Optimization)
Compares sample efficiency of molecular optimization within 10,000 oracle calls
DL2Low-frequency2 Self-reported
20CACHE
Teams submit purchasable compounds and the organizers measure binding: a blind test with real assays
AL5Active0 No ranking
27Polaris / ASAP / OpenADMET
Industry-led benchmark hub, guidelines and blind challenges; its flagship is an antiviral challenge on unpublished project data.
A+CL4Low-frequency0 No ranking
33PoseX
Compares 23 physics, AI-docking and co-folding methods under one protocol, separating self-docking from cross-docking.
BL3Active6 Third-party
44FS-Mol
With only 16-256 labels on a brand-new assay, predict the rest; it fills the few-shot adaptation gap.
BL3Frozen0 Not comparable

Proteins (5)

#BenchmarkTypeEvidenceStatusModelsRanking
02ProteinGym
Unified suite of 200+ public deep mutational scans for protein variant-effect and fitness ranking.
BL2–L3Low-frequency10 Board
17CAFA
Time-delayed blind test of protein function (GO) prediction against experimental annotations added after submission.
AL4–L5Active0 No ranking
26FLIP / FLIP2
Supervised protein fitness benchmark using deliberately shifted splits to test low-resource learning and extrapolation.
BL2–L3Low-frequency2 Not comparable
28Adaptyv competitions / BenchBB
Multi-round prospective protein design evaluation with public data; run by a commercial service provider, not a board to optimise against.
A+BL5Active4 No ranking
40MotifBench
Thirty fixed motif-scaffolding problems judged by one pipeline, with a submission leaderboard; ground truth stays computational.
DL2Active3 Board

Antibodies (4)

#BenchmarkTypeEvidenceStatusModelsRanking
34AbBiBench
Zero-shot scoring correlations evaluated on antibody-antigen complexes, filling a gap in antibody function evaluation.
BL2–L3Active6 Board
35FLAb / FLAb2
Aggregated retrospective antibody developability benchmark with a negative headline result.
BL2–L3Active1 Third-party
36AIntibody
Antibody-side prospective blind test: submitted designs get uniform affinity and developability measurement.
AL5Low-frequency1 Board
43Ginkgo GDPa1
Uniform-assay developability data on 246 clinical antibodies plus a one-off blind test on 80 private held-out antibodies.
A+BL4Low-frequency0 Board

Genomics & RNA (8)

#BenchmarkTypeEvidenceStatusModelsRanking
08Basenji/Enformer test set
The de facto protocol for sequence-to-function models: track prediction on fixed held-out regions.
BL2–L3Active3 Self-reported
11ClinVar hold-out protocol
Separating pathogenic from benign on held-out clinical labels, with each paper choosing its own snapshot and subset.
BL2–L3Active5 Not comparable
13CAGI
Long-running CASP-style blind test of variant interpretation, scored on data unpublished at submission.
AL4–L5Active1 Third-party
16GUE + Nucleotide Transformer benchmark
Short-sequence regulatory-element classification suites for DNA language models; a smoke test, not proof of quality.
BL2–L3Low-frequency0 No ranking
21DREAM5 / BEELINE / CausalBench
Three generations of GRN inference evaluation whose scores mainly reflect gold-standard quality
A+B+CL2–L5Low-frequency0 Not comparable
37TraitGym
Benchmark classifying causal non-coding variants against matched controls.
BL3Low-frequency3 Board
38BEND
Evaluates DNA language models' frozen representations on human-genome functional tasks.
BL2Active0 Board
39RNAGym
Zero-shot evaluation framework for RNA fitness and structure.
BL2Active1 Board

Single-cell & perturbation (2)

#BenchmarkTypeEvidenceStatusModelsRanking
09scIB (+Open Problems)
Default yardstick for single-cell integration, scoring batch removal against biological conservation; continued by Open Problems.
B+CL2Active1 Board
24Virtual Cell Challenge and perturbation benchmarks
The only challenge generating Perturb-seq data for evaluation, yet linear and mean baselines remain hard to beat
A+B+CL2–L4Active9 Board

Molecule generation (2)

#BenchmarkTypeEvidenceStatusModelsRanking
10GuacaMol (+MOSES)
Entry-level molecular generation benchmark; computed oracles only, unmaintained, optimization side superseded by PMO.
DL2Frozen0 No ranking
32CrossDocked2020
The de facto default set for pocket-conditioned molecule generation, scored by docking software.
DL2Frozen1 Not comparable

Text, LLMs & agents (6)

#BenchmarkTypeEvidenceStatusModelsRanking
14PubMedQA
500-question yes/no/maybe test on PubMed abstracts; top scores now sit within sampling noise.
B+EL1Frozen2 Board
18LAB-Bench / LABBench2
Biology research-skill benchmark for LLMs and agents; the multiple-choice original is partly saturated and superseded by LABBench2.
EL1–L2Active3 Board
23BioASQ
The only yearly biomedical retrieval and QA challenge with new expert-written questions and expert grading
A+BL1–L4Active0 Board
29BixBench
Agents write and run code on real data in a container to do bioinformatics analyses, graded on their conclusions.
EL2Active3 Board
41BioML-bench
Agents build and run biomedical ML pipelines end to end, scored against human public-leaderboard results; tested agents average below the human baseline.
EL2Low-frequency1 Self-reported
42BioMysteryBench
Questions on real sequencing data with metadata stripped, graded on the final answer only; authored by a model vendor, with only five public questions.
EL2–L4Active1 Third-party