Overview
| Name | USPTO family (+PaRoutes) · aliases: USPTO-50K, USPTO-MIT, USPTO-480K, MIT-480K, USPTO-FULL, USPTO-STEREO, PaRoutes |
|---|---|
| Full name | USPTO patent reaction dataset family |
| Ambiguous names (not auto-linked) | USPTO |
| Family | Several splits and a route set derived from one patent-text extraction; subsets differ in size, split and preprocessing, so numbers do not transfer across them.
|
| Domain | Small molecules |
| Inclusion | Original 30 |
| Type | B · Static dataset family; PaRoutes adds route sets and scripts |
| Evidence level | L2–L3 · Random-split retrospective; OOD studies reach L3 |
| Publisher | NextMove Software, Novartis, Massachusetts Institute of Technology, AstraZeneca (Mixed) |
| First release | · data Basis: First public release of the patent reaction extraction data (NextMove blog, 2014-02); day not retrieved source |
| Other milestones | |
| Latest update | · data Basis: PaRoutes paper published in Digital Discovery (year only) Newest member: PaRoutes route-benchmark paper in Digital Discovery (preprint reviews 2022-03-18); source data frozen at 2016-09. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | Source data are frozen and no maintainer has announced a new version. |
| Status | ○ Frozen · Source data frozen at 2016; methods papers still report it |
| Scale | Source 3.3M reactions; USPTO-50K about 50k; MIT-480K 479,035; PaRoutes two sets of 10,000 routes.
|
| Wet-lab refresh | Static data · None; patent text mining, not experimental validation. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 5 counted (self 4, modified protocol 1) |
| Unverified fields | † family.members[0], family.members[4], ranking.entries[4], models_reported[2] |
| As of | 2026-11-16 |
What it measures
- Task: Reaction outcome prediction, single-step retrosynthesis and multi-step route search on reactions mined from US patents.
- Truth: Records extracted from patent text; no failed reactions, no yield guarantee, conditions often missing.
- Primary metric: Top-k exact match (canonical SMILES)
- Secondary metrics: Round-trip accuracy; Route solve rate (PaRoutes)
- Version traps: Reaction-class disclosure, test-time augmentation, FULL pretraining and per-paper preprocessing all shift numbers; MIT has mixed and separated settings.
Current ranking
Not comparable
| Metric | top-k exact match (%), canonical SMILES |
|---|---|
| Eval set | USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) |
| Snapshot | |
| Board | No official board |
| Retrieved | 2026-11-16 |
Caveat. No official board; Papers with Code closed. Numbers depend on class disclosure, augmentation and pretraining; ~5k test reactions, 1 point ~50 reactions; few seeds or CIs.
No headline entry: No official board and settings differ across papers, so entries are not comparable and no headline is given.
The entries below use different protocols, so they are listed, not ranked, and the overview table shows no score.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | RetroDFM-R | Withheld: Not comparable: protocols and subsets differ, so no score is shown in the ranking row; per-protocol values stay on the entity page. | top-k exact match (%), canonical SMILES USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) | Self-reported | source Correction: the site cited arXiv 2510.16588 (that is C-SMILES); this model is arXiv 2507.17448; values came via a search summary, not a full read. |
| — | Uni-Mol3 | Withheld: Not comparable: protocols and subsets differ, so no score is shown in the ranking row; per-protocol values stay on the entity page. | top-k exact match (%), canonical SMILES USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) | Self-reported | source Re-verified from the paper; the same tables show it below the Chemformer baseline on 50K retrosynthesis. |
| — | Chemformer | Withheld: Not comparable: protocols and subsets differ, so no score is shown in the ranking row; per-protocol values stay on the entity page. | top-k exact match (%), canonical SMILES USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) | Third-party | source Third-party tabulation (baseline row in the Uni-Mol3 paper), re-verified. |
| — | C-SMILES (not a site model) | Withheld: Not comparable: protocols and subsets differ, so no score is shown in the ranking row; per-protocol values stay on the entity page. | top-k exact match (%), canonical SMILES USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) | Self-reported | source Not a site model; highest 50K top-1 seen this round; the site's known/unknown-class split was not confirmed on the abstract. |
| — | ether0 | Withheld: Incomplete provenance (no url) and a different protocol, so the honesty rule forbids showing a number. | top-k exact match (%), canonical SMILES USPTO-50K, USPTO-MIT/480K, USPTO-FULL; settings differ (reaction class, augmentation, pretraining) | Self-reported No sourced date† | no source link Evaluated on its own reaction set, not an official USPTO test split; not comparable. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 5. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Developer report, official protocol (4)
- Chemformer Self-reported · USPTO-50K; the Uni-Mol3 re-quote is comparison only · evidence: link
- RetroDFM-R Self-reported · USPTO-50K/FULL; with test-time augmentation noted · evidence: link
- SciReasoner Self-reported · USPTO-50K single-step retrosynthesis, matrix note only, unverified †unverified · evidence: no source link
- Uni-Mol3 Self-reported · USPTO-MIT/50K/Condition · evidence: link
Developer report, modified protocol (1)
Caveats
- Random splits leak templates (about 94% of USPTO-50K test reactions match training templates, per one report); exact match penalizes valid alternative routes; no failed reactions are recorded.
- Reaction-class disclosure, test-time augmentation and pretraining corpus can inflate numbers markedly; cross-paper comparison needs aligned settings.
- With about 5k test reactions, recent 1-3 point gains may be within noise (inference); most papers report no CIs.
- Template-free models drop sharply in OOD evaluations; single-step scores are reported to decouple from route-finding success.
- No evidence on lab feasibility of routes; only synthesis can establish it.
How to use it
Use the CC0 figshare source data or a method repo's preprocessed split; PaRoutes ships scripts. Report subset, reaction-class disclosure, augmentation, FULL pretraining, split (prefer new or time splits), seeds with CIs and a template-retrieval baseline; round-trip must name its forward model.
Pairs well with: GuacaMol (+MOSES) · PMO (Practical Molecular Optimization)
Sources
- Chemical reactions from US patents 1976-Sep2016 (figshare, CC0) (archive)
- USPTO-50K source paper (JCIM 2016) (journal)
- USPTO-MIT 480K paper (arXiv 1709.04555) (preprint)
- PaRoutes (Digital Discovery 2022) (journal)
- PaRoutes repository (repository)
- Retrosynthesis Prediction Revisited (NeurIPS 2022 workshop) (third-party paper)
- Retrosynthesis OOD evaluation (arXiv 2403.03960) (third-party paper)
- RetroDFM-R paper (arXiv 2507.17448) (preprint)
Corrections
3 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| ranking.entries[model-retrodfm-r].url | arXiv 2510.16588 cited for RetroDFM-R | arXiv 2507.17448 (2510.16588 is a different paper, C-SMILES) | source | 2026-11-16 |
| ranking.entries[C-SMILES].note | known-class / unknown-class split as stated on the site | abstract gives a USPTO-50K figure and a USPTO-FULL figure; the known/unknown-class split is not confirmed | source | 2026-11-16 |
| first_release.basis | Source data dated 2012 (thesis) | 2014-02, first public data release; 2012 date unverified | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.