Overview
| Name | FS-Mol · aliases: FS-Mol, fsmol-0.1 |
|---|---|
| Full name | FS-Mol: a few-shot molecular activity prediction benchmark |
| Ambiguous names (not auto-linked) | None |
| Family | Not a family entry. Only one data version, fsmol-0.1; the repository and the paper are milestones of one entry, not a family. |
| Domain | Small molecules |
| Inclusion | Supplementary entry |
| Type | B · Fixed task split plus an evaluation protocol |
| Evidence level | L3 · Cross-assay transfer split by target assay, with no blind test |
| Publisher | Microsoft Research, Novartis (Mixed) |
| First release | · paper v1 Basis: NeurIPS 2021 Datasets and Benchmarks paper, 2021-12 (month precision; no arXiv version found). The 2021-08-13 first repository commit is a note only. source |
| Other milestones | |
| Latest update | † partly verified · archived Basis: Repository archive status (the archive date comes from an existing record; the API was rate-limited and it was not re-verified) Repository archived read-only; last code commit 2021-12-08 (a typing fix), maintenance only since. Note: The archive date was not independently re-verified. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | The maintainers have announced no further version and the repository is archived. |
| Status | ○ Frozen · Frozen: the repository has been archived read-only since 2026-10-21; method papers still cite it. |
| Scale | 5,120 assays, 233,786 compounds, 489,133 measurements; 4,938 training, 40 validation and 157 test tasks.
|
| Wet-lab refresh | Static data · No new experimental data; built from ChEMBL27, with no later release by the maintainers. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 0 counted |
| Unverified fields | † latest_update, release_milestones.first_public_evidence, ranking.entries[0] |
| As of | 2026-11-16 |
What it measures
- Task: Pre-train on training tasks, then use 16-256 labelled molecules from a new assay to predict the query set's activity.
- Truth: Activity measurements per assay in ChEMBL27, binarized at that assay's median.
- Primary metric: Delta AUPRC: query AUPRC minus the positive rate
- Secondary metrics: Mean rank; Scores stratified by target class
- Version traps: State the support-set size, whether ECFP or physicochemical features were added, how many tasks qualified, and any extra pre-training data.
Current ranking
Not comparable
| Metric | Delta AUPRC (support set 16-256) |
|---|---|
| Eval set | 157 test tasks (fsmol-0.1, ChEMBL27) |
| Snapshot | |
| Board | No official board |
| Retrieved | 2026-11-16 |
Caveat. No live leaderboard and the repository is archived; later method papers self-report under different support sizes and features, which are not comparable; no site model reports this benchmark.
No headline entry: Not comparable: support sizes and features differ across sources, so no headline number is given and the caveat text is shown instead.
The entries below use different protocols, so they are listed, not ranked, and the overview table shows no score.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | PN (prototypical network) | Withheld: Not comparable: protocols and subsets differ, so no score is shown in the ranking row; per-protocol values stay on the entity page. | Delta AUPRC (support set 16-256) 157 test tasks (fsmol-0.1, ChEMBL27) | Self-reported | source Not a site model; the best baseline in the original paper's table at support size 16. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.
None. No model among the 112 has a recorded pairing with this entry.
Caveats
- Test tasks are enzyme targets only: 125 of 157 are kinases, with 1-20 tasks per other class.
- Split by target, yet protein families and chemical series are not fully separated, so it is closer to neighbourhood interpolation than true out-of-distribution.
- Labels are median-binarized and the positive rate is constrained to 0.30-0.70, unlike the rare positives of real screening.
- Papers use different support-set sizes and features, so numbers are often not directly comparable.
- The official repository is archived read-only and the data do not follow newer ChEMBL releases.
How to use it
Take the data and evaluation code from the official repository, run the official evaluation on the 157 test tasks and tune on the 40 validation tasks. Report delta AUPRC separately for support sizes 16 to 256 and include a random-forest baseline; the number of qualifying tasks varies with support size, so state task coverage. Declare any extra pre-training data and its possible overlap with the test assays. Stronger generalization claims need strict target-family or scaffold splits.
Pairs well with: MoleculeNet (+OGB) · TDC (ADMET + DTI) · Polaris / ASAP / OpenADMET
Sources
- FS-Mol paper (NeurIPS 2021 Datasets and Benchmarks) (paper) ·
- Official repository (archived read-only) (repository) ·
- Few-shot method paper (journal version, 2025) (third-party paper)
- Method paper reporting on this benchmark and on molecular property sets (third-party paper)
Corrections
3 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| first_release | 2021-08-13 | 2021-12 | source | 2026-11-16 |
| first_release_basis | GitHub repo microsoft/FS-Mol initial commit (data + code); NeurIPS 2021 D&B paper Dec 2021; no arXiv version found | NeurIPS 2021 Datasets & Benchmarks paper, 2021-12 (month precision; no arXiv version found). Repo initial commit 2021-08-13 kept as a note only. | source | 2026-11-16 |
| ranking | baseline delta AUPRC values shown as a ranking | status not-comparable: no score or rank is displayed in the ranking row | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.