AI for Bio · Benchmark 06

CASF-2016

Decade-old default test of protein-ligand scoring functions across four powers; frozen and heavily overlapping with training data.

bench-06Type BEvidence L2Small molecules○ Frozen Not comparable中文

Overview

NameCASF-2016 · aliases: CASF-2016, CASF-2013, CASF-2007, CASF-07, Comparative Assessment of Scoring Functions
Full nameComparative Assessment of Scoring Functions 2016
Ambiguous names (not auto-linked)CASF, PDBbind core set
FamilyNot a family entry. CASF-2007, 2013 and 2016 are successive versions of one benchmark, recorded as milestones, not a family.
DomainSmall molecules
InclusionOriginal 30
TypeB · Static dataset suite with official scoring scripts
Evidence levelL2 · Retrospective, not blind; heavy train-test overlap
PublisherShanghai Institute of Organic Chemistry, CAS (PDBbind-CN team) (Academic lab)
First release
Basis: CASF-2007 paper online in JCIM; the benchmark is named 'CASF-2007' but was first published in 2009
Note: 2007 is the version label, not the release date source
Other milestones
  • Journal online: · CASF-2007 paper online in JCIM source
  • Official announcement: · CASF-2016 update paper online in JCIM (current version) source
Latest update · no later event
Basis: CASF-2016 paper online date in JCIM; absence of a newer version rests on web search, the official site was not fetched
CASF-2016 update paper published online in JCIM; no later CASF version found. source
The most recent sourced event, not a claim that nothing changed since.
Next expectedThe publisher has announced no version after CASF-2016.
Status○ Frozen · Frozen since 2016; being replaced by leakage-controlled splits
Scale285 complexes (57 target clusters x 5 ligands), about 100 docking decoys per complex.
  • Complexes: 285 (CASF-2016)
  • Target clusters: 57 (CASF-2016)
  • Ligands per cluster: 5 (CASF-2016)
  • Docking decoys per complex: ~100 (CASF-2016)
Wet-lab refreshStatic data · None; reuses PDBbind crystal structures and literature Kd/Ki.
Own wet lab needed to evaluate a new modelYes, to claim real generalization
Models reported0 counted
Unverified fieldsNone
As of2026-11-16

What it measures

  • Task: Scoring, ranking, docking and screening power of protein-ligand scoring functions on the PDBbind core set.
  • Truth: PDBbind crystal structures and literature Kd/Ki mixed across labs and assays; screening negatives are assumed.
  • Primary metric: Scoring power: Pearson R, RMSE
  • Secondary metrics: Ranking power: within-cluster Spearman ρ; Docking power: top-1 success (RMSD<2Å); Screening power: EF1%, success rate
  • Version traps: Only the test set is fixed; state the PDBbind version and whether CleanSplit or LP-PDBBind was used; never mix CASF-2013 and 2016.

Current ranking

Not comparable

MetricScoring power Pearson R / RMSE
Eval setCASF-2016 core set (285)
Snapshot stale
BoardNo official board
Retrieved2026-11-16

Caveat. No site model reports CASF-2016; no official board; ~49% of test complexes have similar training pairs, so splits dominate; older high scores used leaky splits, unverified.

No headline entry: Not comparable: no official board and scores depend mainly on the training split, so no headline number.

The entries below use different protocols, so they are listed, not ranked, and the overview table shows no score.

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
—GEMS (trained on PDBbind CleanSplit)Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page.Scoring power Pearson R / RMSE
CASF-2016 core set (285)
Self-reported
source Not a site model; self-reported, trained on CleanSplit
—GenScore (CleanSplit retrain)Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page.Scoring power Pearson R / RMSE
CASF-2016 core set (285)
Third-party
source Not a site model; retrained on CleanSplit in a leakage assessment
—Pafnucy (CleanSplit retrain)Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page.Scoring power Pearson R / RMSE
CASF-2016 core set (285)
Third-party
source Not a site model; retrained on CleanSplit in a leakage assessment
—5-nearest-neighbour search baseline (no training)Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page.Scoring power Pearson R / RMSE
CASF-2016 core set (285)
Third-party
source Not a model: retrieval baseline averaging the most similar training complexes

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.

None. No model among the 112 has a recorded pairing with this entry.

Caveats

  1. Test and PDBbind training sets overlap heavily: a no-training retrieval baseline approaches older deep models; docking power is saturated and screening negatives are unmeasured cross-pairs.
  2. A 2025 leakage study reports similar training pairs for about 49% of test complexes; most models drop after de-leaking.
  3. Ranking uses only 57 five-ligand groups and screening only 285 ligands, so noise is large; no official intervals or pose-validity checks.
  4. Boltz-2 official evaluation docs do not list CASF; no recent AI model report on it was found this round.
  5. PDBbind data now sits behind a licence gate whose terms were not verified on official pages.

How to use it

Register on the PDBbind site, download the CASF-2016 package and run the four powers with official scripts; no wet lab, retrospective only. Train on CleanSplit or LP-PDBBind, add a retrieval baseline and a time-split set, state the training version. Pair with PoseBusters, PLINDER, DUD-E, CACHE.

Pairs well with: PoseBusters · PLINDER / PINDER / Runs N' Poses · DUD-E / LIT-PCBA · CACHE

Sources

  1. CASF-2016 paper (JCIM 2019) (journal)
  2. CASF-2007 paper (JCIM 2009) (journal)
  3. CASF-2013 paper (JCIM 2014) (journal)
  4. PDBbind/CASF protocol (Nature Protocols 2018) (journal)
  5. PDBbind CleanSplit leakage study (Nat Mach Intell 2025) (third-party paper)
  6. LP-PDBBind re-split (arXiv 2308.09639) (third-party paper)
  7. Boltz official evaluation docs (repository)
  8. PDBbind licensing discussion (arXiv 2411.01223) (third-party paper)

Corrections

1 correction
FieldWasNowBasisFound
first_release2007 (start year)2009-04-09source2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.