Overview
| Name | CASF-2016 · aliases: CASF-2016, CASF-2013, CASF-2007, CASF-07, Comparative Assessment of Scoring Functions |
|---|---|
| Full name | Comparative Assessment of Scoring Functions 2016 |
| Ambiguous names (not auto-linked) | CASF, PDBbind core set |
| Family | Not a family entry. CASF-2007, 2013 and 2016 are successive versions of one benchmark, recorded as milestones, not a family. |
| Domain | Small molecules |
| Inclusion | Original 30 |
| Type | B · Static dataset suite with official scoring scripts |
| Evidence level | L2 · Retrospective, not blind; heavy train-test overlap |
| Publisher | Shanghai Institute of Organic Chemistry, CAS (PDBbind-CN team) (Academic lab) |
| First release | Basis: CASF-2007 paper online in JCIM; the benchmark is named 'CASF-2007' but was first published in 2009 Note: 2007 is the version label, not the release date source |
| Other milestones | |
| Latest update | · no later event Basis: CASF-2016 paper online date in JCIM; absence of a newer version rests on web search, the official site was not fetched CASF-2016 update paper published online in JCIM; no later CASF version found. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | The publisher has announced no version after CASF-2016. |
| Status | ○ Frozen · Frozen since 2016; being replaced by leakage-controlled splits |
| Scale | 285 complexes (57 target clusters x 5 ligands), about 100 docking decoys per complex.
|
| Wet-lab refresh | Static data · None; reuses PDBbind crystal structures and literature Kd/Ki. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 0 counted |
| Unverified fields | None |
| As of | 2026-11-16 |
What it measures
- Task: Scoring, ranking, docking and screening power of protein-ligand scoring functions on the PDBbind core set.
- Truth: PDBbind crystal structures and literature Kd/Ki mixed across labs and assays; screening negatives are assumed.
- Primary metric: Scoring power: Pearson R, RMSE
- Secondary metrics: Ranking power: within-cluster Spearman ρ; Docking power: top-1 success (RMSD<2Å); Screening power: EF1%, success rate
- Version traps: Only the test set is fixed; state the PDBbind version and whether CleanSplit or LP-PDBBind was used; never mix CASF-2013 and 2016.
Current ranking
Not comparable
| Metric | Scoring power Pearson R / RMSE |
|---|---|
| Eval set | CASF-2016 core set (285) |
| Snapshot | stale |
| Board | No official board |
| Retrieved | 2026-11-16 |
Caveat. No site model reports CASF-2016; no official board; ~49% of test complexes have similar training pairs, so splits dominate; older high scores used leaky splits, unverified.
No headline entry: Not comparable: no official board and scores depend mainly on the training split, so no headline number.
The entries below use different protocols, so they are listed, not ranked, and the overview table shows no score.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | GEMS (trained on PDBbind CleanSplit) | Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page. | Scoring power Pearson R / RMSE CASF-2016 core set (285) | Self-reported | source Not a site model; self-reported, trained on CleanSplit |
| — | GenScore (CleanSplit retrain) | Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page. | Scoring power Pearson R / RMSE CASF-2016 core set (285) | Third-party | source Not a site model; retrained on CleanSplit in a leakage assessment |
| — | Pafnucy (CleanSplit retrain) | Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page. | Scoring power Pearson R / RMSE CASF-2016 core set (285) | Third-party | source Not a site model; retrained on CleanSplit in a leakage assessment |
| — | 5-nearest-neighbour search baseline (no training) | Withheld: not-comparable: protocols/subsets differ, so no score is displayed in the ranking row; the per-protocol values live on the entity page. | Scoring power Pearson R / RMSE CASF-2016 core set (285) | Third-party | source Not a model: retrieval baseline averaging the most similar training complexes |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.
None. No model among the 112 has a recorded pairing with this entry.
Caveats
- Test and PDBbind training sets overlap heavily: a no-training retrieval baseline approaches older deep models; docking power is saturated and screening negatives are unmeasured cross-pairs.
- A 2025 leakage study reports similar training pairs for about 49% of test complexes; most models drop after de-leaking.
- Ranking uses only 57 five-ligand groups and screening only 285 ligands, so noise is large; no official intervals or pose-validity checks.
- Boltz-2 official evaluation docs do not list CASF; no recent AI model report on it was found this round.
- PDBbind data now sits behind a licence gate whose terms were not verified on official pages.
How to use it
Register on the PDBbind site, download the CASF-2016 package and run the four powers with official scripts; no wet lab, retrospective only. Train on CleanSplit or LP-PDBBind, add a retrieval baseline and a time-split set, state the training version. Pair with PoseBusters, PLINDER, DUD-E, CACHE.
Pairs well with: PoseBusters · PLINDER / PINDER / Runs N' Poses · DUD-E / LIT-PCBA · CACHE
Sources
- CASF-2016 paper (JCIM 2019) (journal)
- CASF-2007 paper (JCIM 2009) (journal)
- CASF-2013 paper (JCIM 2014) (journal)
- PDBbind/CASF protocol (Nature Protocols 2018) (journal)
- PDBbind CleanSplit leakage study (Nat Mach Intell 2025) (third-party paper)
- LP-PDBBind re-split (arXiv 2308.09639) (third-party paper)
- Boltz official evaluation docs (repository)
- PDBbind licensing discussion (arXiv 2411.01223) (third-party paper)
Corrections
1 correction
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| first_release | 2007 (start year) | 2009-04-09 | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.