AI for Bio · Benchmark 27

Polaris / ASAP / OpenADMET

Industry-led benchmark hub, guidelines and blind challenges; its flagship is an antiviral challenge on unpublished project data.

bench-27Type A+CEvidence L4Small molecules◐ Low-frequency No ranking中文

Overview

NamePolaris / ASAP / OpenADMET · aliases: Polaris Hub, polaris-lib, ASAP-Polaris-OpenADMET antiviral challenge, antiviral-drug-discovery-2025
Full namePolaris benchmarking hub and its hosted ASAP antiviral blind challenge
Ambiguous names (not auto-linked)Polaris, ASAP, OpenADMET
FamilyNot a family entry. Single hub entry; only Polaris-hosted challenges count, and OpenADMET's own challenges are not members.
DomainSmall molecules
InclusionOriginal 30
TypeA C · Blind challenge plus hosting hub and method guidelines
Evidence levelL4 · Hidden blind test on real project data; no L5
PublisherPolaris (industry-led steering committee), Valence Labs, ASAP Discovery Consortium, OpenADMET (Mixed)
First release · code release
Basis: First polaris-lib PyPI release 0.2.0 (public hub launch date not verified) source
Other milestones
  • First public evidence: No sourced date† · Public hub launch date not found source
  • Code release: · polaris-lib 0.2.0 released on PyPI source
  • Challenge opens: † · Antiviral challenge opened (from the existing record, not independently re-verified) source
Latest update† partly verified · new version
Basis: Latest polaris-lib release on PyPI
polaris-lib 0.13.0 released (last release); last main-branch commit 2025-07-30; Polaris blog has nothing after 2025-01-29.
Note: OpenADMET's CYP (winners 2026-11-05) and PXR challenges ran off the hub; note only source
The most recent sourced event, not a claim that nothing changed since.
Next expectedPolaris has announced no date for its next hosted challenge.
Status◐ Low-frequency · Hub online; few Polaris-run events since 2025
ScaleAntiviral challenge: 3 tracks, 66 participants, 381 submissions; later OpenADMET challenges are not counted here.
  • Tracks (pose, potency, ADMET): 3 (antiviral challenge 2025)
  • Independent participants: 66 (antiviral challenge 2025 (summary paper))
  • Submissions: 381 (antiviral challenge 2025 (summary paper))
  • Final-board submissions: 93 (antiviral challenge 2025 (summary paper))
Wet-lab refreshFresh experiments each round · Hosted-challenge truth is unpublished project assays and new crystal structures, unblinded afterwards; new truth only by entering the next challenge.
Own wet lab needed to evaluate a new modelNo, run it as a download
Models reported0 counted (cited only 1, unconfirmed 2)
Unverified fields† latest_update, release_milestones.first_public_evidence, release_milestones.challenge_open, models_reported[1], models_reported[2]
As of2026-11-16

What it measures

  • Task: Prospective prediction on unpublished industry project data: ADMET endpoints, potency and ligand poses.
  • Truth: Consortium-measured biochemical and ADMET assays and crystal structures, hidden until unblinding.
  • Primary metric: Pose success (RMSD<2Å); MAE for potency and ADMET
  • Secondary metrics: Mean RMSD; Kendall τ ranking; Bootstrap plus Tukey HSD ranking
  • Version traps: Competition page and summary paper use different pose metrics, so cite the source; after unblinding the set is static and not comparable to blind results.

Current ranking

No ranking

MetricAntiviral challenge: pose success rate, MAE for potency and ADMET
Eval setASAP-Polaris antiviral challenge 2025 (Polaris-hosted); OpenADMET CYP/ExpansionRx/PXR boards out of scope
Snapshot† partly verified
Boardhttps://polaris-hub.vercel.app/competitions/asap-discovery/antiviral-drug-discovery-2025
Retrieved2026-11-16

Caveat. Only Polaris-hosted challenges count; their boards are unreadable and the summary paper anonymises entries, so no ranks; OpenADMET boards such as CYP are not this entry's ranking.

No headline entry: Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET CYP board is outside this entry.

No ranking entries recorded.

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.

Cited as comparison only (1, not counted)

  • Chemprop v2 Third-party · Anonymous baseline in the organiser report; comparison only †partly verified · evidence: link

Unconfirmed (2, not counted)

  • Beacon (Inductive Bio) Unconfirmed source · Press release only; organiser anonymises entries; model identity unverified Resolution: Two reasons. (1) Identity: organiser boards are pseudonymous, so the public model version cannot be matched to entries; first-place claims rest on a company press release, only one corroborated by a third-party write-up. (2) Scope: this entry covers Polaris-hosted challenges only, and ExpansionRx and PXR were run by OpenADMET off the hub. Not established. †disputed · evidence: link
  • Boltz-2 Unconfirmed source · Site page says it matched the antiviral top 5 without fine-tuning; no numbers; participation not openly verified low confidence †disputed · evidence: no source link

Caveats

  1. Each challenge covers a single project and chemical series; once unblinded it becomes a static set; no evidence linking scores to wet-lab decisions.
  2. The summary paper anonymises all entries, so public material cannot map ranks to companies or models; rank claims mostly come from press releases.
  3. The competition page sets no formal rules, so external and private data were allowed; it notes the bootstrap may underestimate variance.
  4. OpenADMET's CYP, ExpansionRx and PXR challenges ran outside the Polaris hub and are notes only, not this entry's updates or rankings.

How to use it

Unblinded sets run statically with no wet lab: install polaris-lib, download the unblinded data, fit a fingerprint plus GBM baseline and apply Tukey tests; check poses with PoseBusters. A true blind test needs the next challenge. Use as external validation for TDC and compare with CACHE.

Pairs well with: TDC (ADMET + DTI) · PoseBusters · CACHE · MoleculeNet (+OGB)

Sources

  1. Antiviral challenge competition page (official site)
  2. Antiviral challenge summary paper (JCIM 2026) (journal)
  3. Summary paper open version (University of Cambridge repository, CC BY) (archive)
  4. polaris-lib (PyPI) (package registry)
  5. Polaris client repository (repository)
  6. Polaris small-molecule guidelines (official site)
  7. Correspondence on industry-led blind evaluation (Nature Machine Intelligence 2024) (journal)
  8. OpenADMET blind challenges page (off-hub events) (official site)
  9. OpenADMET CYP challenge results post (note only) (company blog)

Corrections

7 corrections
FieldWasNowBasisFound
latest_update2026-11-052025-06-06source2026-11-16
latest_update_descOpenADMET CYP Inhibition Blind Challenge winners announced (2 months, 6,000+ submissions); polaris-lib last release 0.13.0 on 2025-06-06.polaris-lib 0.13.0 released 2025-06-06 (last release object); last main-branch commit 2025-07-30 (multi-test-set benchmark support); Polaris blog shows nothing after 2025-01-29. Note, not a Polaris update: OpenADMET ran the CYP Inhibition Blind Challenge (winners posted 2026-11-05) and the PXR challenge on Hugging Face, outside the Polaris hub.source2026-11-16
latest_update_kindleaderboard-entrynew-versionsource2026-11-16
rankingboard: OpenADMET CYP Inhibition Blind Challenge final leaderboards (top-1 per track, Boltz-2-named participant entry)no-ranking; entries []source2026-11-16
models_reported (Beacon)Beacon first in three blind challenges (self-reported)disputed-or-unconfirmedsource2026-11-16
statusactive (new challenge results published in 2026-11)low-frequencysource2026-11-16
scaleExpansionRx 370+ teams; PXR 513 test compounds; CYP 750 compoundsantiviral challenge only: 3 tracks, 66 participants, 381 submissionssource2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.