Overview
| Name | Polaris / ASAP / OpenADMET · aliases: Polaris Hub, polaris-lib, ASAP-Polaris-OpenADMET antiviral challenge, antiviral-drug-discovery-2025 |
|---|---|
| Full name | Polaris benchmarking hub and its hosted ASAP antiviral blind challenge |
| Ambiguous names (not auto-linked) | Polaris, ASAP, OpenADMET |
| Family | Not a family entry. Single hub entry; only Polaris-hosted challenges count, and OpenADMET's own challenges are not members. |
| Domain | Small molecules |
| Inclusion | Original 30 |
| Type | A C · Blind challenge plus hosting hub and method guidelines |
| Evidence level | L4 · Hidden blind test on real project data; no L5 |
| Publisher | Polaris (industry-led steering committee), Valence Labs, ASAP Discovery Consortium, OpenADMET (Mixed) |
| First release | · code release Basis: First polaris-lib PyPI release 0.2.0 (public hub launch date not verified) source |
| Other milestones | |
| Latest update | † partly verified · new version Basis: Latest polaris-lib release on PyPI polaris-lib 0.13.0 released (last release); last main-branch commit 2025-07-30; Polaris blog has nothing after 2025-01-29. Note: OpenADMET's CYP (winners 2026-11-05) and PXR challenges ran off the hub; note only source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | Polaris has announced no date for its next hosted challenge. |
| Status | ◐ Low-frequency · Hub online; few Polaris-run events since 2025 |
| Scale | Antiviral challenge: 3 tracks, 66 participants, 381 submissions; later OpenADMET challenges are not counted here.
|
| Wet-lab refresh | Fresh experiments each round · Hosted-challenge truth is unpublished project assays and new crystal structures, unblinded afterwards; new truth only by entering the next challenge. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 0 counted (cited only 1, unconfirmed 2) |
| Unverified fields | † latest_update, release_milestones.first_public_evidence, release_milestones.challenge_open, models_reported[1], models_reported[2] |
| As of | 2026-11-16 |
What it measures
- Task: Prospective prediction on unpublished industry project data: ADMET endpoints, potency and ligand poses.
- Truth: Consortium-measured biochemical and ADMET assays and crystal structures, hidden until unblinding.
- Primary metric: Pose success (RMSD<2Å); MAE for potency and ADMET
- Secondary metrics: Mean RMSD; Kendall τ ranking; Bootstrap plus Tukey HSD ranking
- Version traps: Competition page and summary paper use different pose metrics, so cite the source; after unblinding the set is static and not comparable to blind results.
Current ranking
No ranking
| Metric | Antiviral challenge: pose success rate, MAE for potency and ADMET |
|---|---|
| Eval set | ASAP-Polaris antiviral challenge 2025 (Polaris-hosted); OpenADMET CYP/ExpansionRx/PXR boards out of scope |
| Snapshot | † partly verified |
| Board | https://polaris-hub.vercel.app/competitions/asap-discovery/antiviral-drug-discovery-2025 |
| Retrieved | 2026-11-16 |
Caveat. Only Polaris-hosted challenges count; their boards are unreadable and the summary paper anonymises entries, so no ranks; OpenADMET boards such as CYP are not this entry's ranking.
No headline entry: Polaris-hosted boards are JS-rendered and unreadable, and the summary paper anonymises entries; the OpenADMET CYP board is outside this entry.
No ranking entries recorded.
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Cited as comparison only (1, not counted)
- Chemprop v2 Third-party · Anonymous baseline in the organiser report; comparison only †partly verified · evidence: link
Unconfirmed (2, not counted)
- Beacon (Inductive Bio) Unconfirmed source · Press release only; organiser anonymises entries; model identity unverified Resolution: Two reasons. (1) Identity: organiser boards are pseudonymous, so the public model version cannot be matched to entries; first-place claims rest on a company press release, only one corroborated by a third-party write-up. (2) Scope: this entry covers Polaris-hosted challenges only, and ExpansionRx and PXR were run by OpenADMET off the hub. Not established. †disputed · evidence: link
- Boltz-2 Unconfirmed source · Site page says it matched the antiviral top 5 without fine-tuning; no numbers; participation not openly verified low confidence †disputed · evidence: no source link
Caveats
- Each challenge covers a single project and chemical series; once unblinded it becomes a static set; no evidence linking scores to wet-lab decisions.
- The summary paper anonymises all entries, so public material cannot map ranks to companies or models; rank claims mostly come from press releases.
- The competition page sets no formal rules, so external and private data were allowed; it notes the bootstrap may underestimate variance.
- OpenADMET's CYP, ExpansionRx and PXR challenges ran outside the Polaris hub and are notes only, not this entry's updates or rankings.
How to use it
Unblinded sets run statically with no wet lab: install polaris-lib, download the unblinded data, fit a fingerprint plus GBM baseline and apply Tukey tests; check poses with PoseBusters. A true blind test needs the next challenge. Use as external validation for TDC and compare with CACHE.
Pairs well with: TDC (ADMET + DTI) · PoseBusters · CACHE · MoleculeNet (+OGB)
Sources
- Antiviral challenge competition page (official site)
- Antiviral challenge summary paper (JCIM 2026) (journal)
- Summary paper open version (University of Cambridge repository, CC BY) (archive)
- polaris-lib (PyPI) (package registry)
- Polaris client repository (repository)
- Polaris small-molecule guidelines (official site)
- Correspondence on industry-led blind evaluation (Nature Machine Intelligence 2024) (journal)
- OpenADMET blind challenges page (off-hub events) (official site)
- OpenADMET CYP challenge results post (note only) (company blog)
Corrections
7 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| latest_update | 2026-11-05 | 2025-06-06 | source | 2026-11-16 |
| latest_update_desc | OpenADMET CYP Inhibition Blind Challenge winners announced (2 months, 6,000+ submissions); polaris-lib last release 0.13.0 on 2025-06-06. | polaris-lib 0.13.0 released 2025-06-06 (last release object); last main-branch commit 2025-07-30 (multi-test-set benchmark support); Polaris blog shows nothing after 2025-01-29. Note, not a Polaris update: OpenADMET ran the CYP Inhibition Blind Challenge (winners posted 2026-11-05) and the PXR challenge on Hugging Face, outside the Polaris hub. | source | 2026-11-16 |
| latest_update_kind | leaderboard-entry | new-version | source | 2026-11-16 |
| ranking | board: OpenADMET CYP Inhibition Blind Challenge final leaderboards (top-1 per track, Boltz-2-named participant entry) | no-ranking; entries [] | source | 2026-11-16 |
| models_reported (Beacon) | Beacon first in three blind challenges (self-reported) | disputed-or-unconfirmed | source | 2026-11-16 |
| status | active (new challenge results published in 2026-11) | low-frequency | source | 2026-11-16 |
| scale | ExpansionRx 370+ teams; PXR 513 test compounds; CYP 750 compounds | antiviral challenge only: 3 tracks, 66 participants, 381 submissions | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.