AI for Bio · Benchmark 20

CACHE

Teams submit purchasable compounds and the organizers measure binding: a blind test with real assays

bench-20Type AEvidence L5Small molecules● Active No ranking中文

Overview

NameCACHE · aliases: CACHE, CACHE challenge, CACHE #1, CACHE #5, CACHE #8
Full nameCritical Assessment of Computational Hit-finding Experiments
Ambiguous names (not auto-linked)DREAM x CACHE, Target 2035
FamilyNot a family entry. Successive rounds of one challenge rather than a family entry; per-round volumes are stored in the scale items.
DomainSmall molecules (also: Structure & co-folding)
InclusionOriginal 30
TypeA · Periodic prospective blind test, one target per round
Evidence levelL5 · Prospective wet lab; no independent replication
PublisherStructural Genomics Consortium, Conscience (Nonprofit institute)
First release† partly verified
Basis: Organizer page announcing challenge #1 targets and open participation; no same-day archive
Note: The 2022-01-31 application deadline is kept as a note only and is never a first release. source
Other milestones
  • First public evidence: · Application deadline for challenge #1, recorded as a note and not a release source
  • Official announcement: † · Organizer announcement of open participation for challenge #1 source
Latest update† partly verified · data
Basis: Host organization's press release PDF on the challenge #5 results, uploaded in August 2026
The host organization issued a press release on the challenge #5 results (PDF uploaded August 2026); DREAM x CACHE Target 2035 launched in June 2026.
Note: The exact press-release day was not seen; the site note says 2026-08-18. source
The most recent sourced event, not a claim that nothing changed since.
Next expected† partly verified · new round
Basis: Host press release says challenge #9 is expected late 2026; DREAM x CACHE applications close 2026-11-30 source
Status● Active · Active: challenge #8 under way, challenge #5 results published 2026-08
ScaleRounds #1-#8, one target each and up to 100 compounds per team; round #5 totalled 2,031 compounds and round #6 1,344.
  • Round #1 compounds, 23 teams: 1955 compounds (CACHE #1)
  • Round #5 compounds in total, 24 teams: 2031 compounds (CACHE #5)
  • Round #5, stage 1: 1455 compounds (CACHE #5)
  • Round #5, stage 2, 15 participants: 576 compounds (CACHE #5)
  • Round #6 compounds, 18 participants: 1344 compounds (CACHE #6)
  • Per-team submission cap: 100 compounds
Wet-lab refreshFresh experiments each round · Yes: about two rounds a year, 1,300-2,000 compounds plus 300-600 analogues measured in laboratories each round.
Own wet lab needed to evaluate a new modelNo, run it as a download
Models reported0 counted
Unverified fields† first_release, latest_update, next_expected, ranking.entries[0], ranking.entries[1], ranking.snapshot_date
As of2026-11-16

What it measures

  • Task: One target per round: teams submit up to 100 purchasable compounds, the organizers screen and orthogonally confirm them, and a second stage takes analogues.
  • Truth: Newly measured binding data from the organizers (SPR and similar) plus functional or orthogonal assays in stage 2.
  • Primary metric: Experimentally confirmed hits and hit rate
  • Secondary metrics: Confirmed analogues and series in stage 2; Affinity range of the hits
  • Version traps: Targets, assays and hit criteria differ by round (round #1 counted KD below 150 uM as a hit), so hit rates are not comparable across rounds.

Current ranking

No ranking

MetricExperimentally confirmed hits and hit rate (SPR, dose-response, functional assays); no single score
Eval setCACHE challenges #1-#8, one target per round, up to 100 purchasable compounds per team
Snapshot† partly verified
Boardhttps://cache-challenge.org/
Retrieved2026-11-16

Caveat. CACHE prospectively evaluates whole team workflows rather than single models and publishes hit counts only; per-round hit rates of 0.7-4% leave samples too small for significance testing.

No headline entry: The organizers publish no ranking and never name the models inside each workflow, so no number can serve as a headline.

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
—CACHE #5 (MCHR1) round results, workflow level rather than model levelWithheld: The page has no date and the organizers publish no rank, so the honesty rule keeps this out of the ranking row.Experimentally confirmed hits and hit rate (SPR, dose-response, functional assays); no single score
CACHE challenges #1-#8, one target per round, up to 100 purchasable compounds per team
Board
No sourced date†
source Not a site model. The official results page gives hit counts by workflow id, discloses no models and publishes no rank.
—Boltz-2Withheld: No number and no date, so nothing is shown in the ranking row.Experimentally confirmed hits and hit rate (SPR, dose-response, functional assays); no single score
CACHE challenges #1-#8, one target per round, up to 100 purchasable compounds per team
Third-party
No sourced date†
source The Boltz-2 paper reports no CACHE participation and CACHE result pages do not name the models inside each workflow, so its use is not disclosed.

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.

None. No model among the 112 has a recorded pairing with this entry.

Mentioned, not a pairing

  • · Its paper reports no CACHE result and the official pages do not list the models used by each workflow, so there is no pairing. no source link

Caveats

  1. It evaluates whole team workflows, not single models, and never discloses which models a workflow used.
  2. Each round yields only tens of hits at 0.7-4%, and without significance tests ML and physics methods cannot be compared statistically.
  3. Most hits are weak binders in the tens of micromolar, and binding is not functional activity or developability.
  4. Biases: compounds come from a single supplier, entry is voluntary, and assay sensitivity limits detection of weak binders.
  5. Rounds #7 and #8 have no published results yet, and peer-reviewed papers for rounds #4-#6 lag behind.

How to use it

A real blind test means entering the next round; compound-level data from released rounds can serve as a retrospective set but is no longer blind for models with later training cutoffs. Report the round, target, hit criterion and assay, and separate stage 1 from stage 2. A real generalization claim needs prospective measurement, whose cost the organizers carry in that round.

Pairs well with: PoseBusters · PLINDER / PINDER / Runs N' Poses · DUD-E / LIT-PCBA · Polaris / ASAP / OpenADMET

Sources

  1. CACHE official site (official site)
  2. Official results page for challenge #5 (official site)
  3. Host organization press release on the challenge #5 results (press) · †
  4. Rules page for challenge #7 (official site)
  5. Challenge #1 results preprint (preprint)
  6. Paper on a challenge #3 participant workflow (journal)

Corrections

5 corrections
FieldWasNowBasisFound
first_release2022-01-312021-12-01source2026-11-16
first_release_basischallenge #1 application deadline (launch date not stated on site)official announcement of challenge #1 targets and open participation, 2021-12-01 (organiser page). This is the announced arrangement; no same-day archive of the live opening was obtained, so the day stays flagged. Application deadline 2022-01-31 is stored as a note only.source2026-11-16
members[{"member": "CACHE #1 (deadline)", "date": "2022-01-31"}, {"member": "CACHE #8 (deadline)", "date": "2026-04-30"}][{"member": "CACHE #1 announced open (2021-12-01)", "date": "2021-12-01"}, {"member": "CACHE #1 application deadline", "date": "2022-01-31"}, {"member": "CACHE #8 application deadline", "date": "2026-04-30"}]source2026-11-16
scaleround #5: 2,031 compounds; round #6: 1,344unchanged, with provenance: round #5's 2,031 is the sum of stage 1 (1,455 compounds, 24 teams) and stage 2 (576); round #6 is 1,344 compounds from 18 participantssource2026-11-16
ranking.statusround results shown as a ranking rowno-ranking: hit counts at workflow level, with the undated results page withheld from displaysource2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.