Overview
| Name | MotifBench · aliases: MotifBench |
|---|---|
| Full name | The name is already the full name; no expanded form is given by the sources. |
| Ambiguous names (not auto-linked) | None |
| Family | Not a family entry. One fixed problem set and one leaderboard; no family members. |
| Domain | Proteins (also: Molecule generation) |
| Inclusion | Supplementary entry |
| Type | D · Computational-oracle problem set with a submission leaderboard |
| Evidence level | L2 · Computational self-consistency: ProteinMPNN redesign plus ESMFold refolding |
| Publisher | Multi-lab academic collaboration (MotifBench maintainers) (Academic lab) |
| First release | · paper v1 Basis: arXiv 2502.12479 v1 (repository created 2024-06-14, not used as first release) source |
| Other milestones | |
| Latest update | · board entry Basis: Leaderboard commits in the repository Leaderboard updated: RFD4-Proteina results added 2026-11-06, reformatted and consolidated into one entry on 2026-11-10/13. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | The maintainers have announced no next version or round. |
| Status | ● Active · Active: the leaderboard has new entries from 2025-02 through 2026-11. |
| Scale | Thirty problems in three groups of ten by number of motif segments; 100 scaffolds required per problem.
|
| Wet-lab refresh | Static data · Purely computational evaluation; reference scaffolds come from experimental structures, but the evaluation itself has no wet lab. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 3 counted (board 3) |
| Unverified fields | † ranking.entries[0].date, ranking.entries[1].date, ranking.entries[2].date, ranking.entries[3].date |
| As of | 2026-11-16 |
What it measures
- Task: Given a motif, generate fixed-length scaffolds; 100 per problem, judged by sequence redesign and refolding.
- Truth: Computational oracle: geometric agreement between the refolded structure and the input motif.
- Primary metric: MotifBench score (0-100)
- Secondary metrics: Unique solutions after clustering; Novelty (TM-score against the PDB); Per-scaffold success rate
- Version traps: State whether a number comes from the paper or the leaderboard, whether the indexed setting was used, and whether refolding used ESMFold or AlphaFold2.
Current ranking
Board
| Metric | MotifBench score (0-100) |
|---|---|
| Eval set | 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline |
| Snapshot | |
| Board | https://github.com/blt2114/MotifBench |
| Retrieved | 2026-11-16 |
Caveat. Ranks follow the board table order (sorted by score, 15 entries); entries are submitter-run, not independently re-run. The top entry is not a site model.
Shown in the overview table: Board
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| 1 | RFD4-Proteina (indexed + medium thinking) | 55.06 (± 1.35) | MotifBench score (0-100) 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline | Board † | source Not a site model; overall top-1. |
| 2 | RFdiffusion3 (indexed) | 43.52 (± 1.35) | MotifBench score (0-100) 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline | Board † | source Submitter-run entry; best site model. |
| 5 | La-Proteina | 29.75 (± 0.51) | MotifBench score (0-100) 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline | Board † | source Submitted by another generative-model team. |
| 15 | ESM3 | 11.38 | MotifBench score (0-100) 30 motif-scaffolding problems, official ProteinMPNN + ESMFold pipeline | Board † | source Last of the 15 entries. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 3. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Official leaderboard or results (3)
- ESM3 Board MotifBench submission leaderboard (2025-05 submission) · Entry on the MotifBench submission board (11.38, 2025-05). · evidence: link
- La-Proteina Board · The board lists this model (29.75, 2026-07); the Chinese model page says there is no score, which is wrong. Resolution: The board lists La-Proteina (29.75, 2026-07), so the relation is official participation; the model page claiming no score is the error and must be fixed. · evidence: link
- RFdiffusion2 / RFdiffusion3 Board official board entry (RFdiffusion3, 2026-08 snapshot) · The board lists RFdiffusion3 (43.52, 2026-08); the Chinese model page says no entry was found, which is wrong. Resolution: The board and this record agree in listing RFdiffusion3 (43.52, 2026-08) as official participation; the Chinese model page must be fixed. · evidence: link
Mentioned, not a pairing
- · The Proteina-Complexa method paper does not mention this problem set; the earlier candidate list was wrong. no source link
Caveats
- Scores reflect in-silico self-consistency only, not expression, folding or function.
- Replicate standard deviations are about 0.39-0.47, and the authors treat gaps under about one point as unreliable.
- Paper and leaderboard give different RFdiffusion baselines (28.05 vs 21.95), so cite the protocol.
- Refolding with MSA-free AlphaFold2 instead of ESMFold drops the baseline from 28.1 to 22.5.
- Experimental reference scaffolds pass the evaluation in only 20 of 30 problems, which points to a limit of the evaluation itself.
How to use it
Take the 30 problems and the official pipeline from the repository, generate 100 fixed-length scaffolds per problem, score with the official scripts and submit to the leaderboard; about 36 GPU hours. Report whether the protocol is the paper's or the board's, whether the indexed setting was used, which refolder was chosen and how many replicates; gaps under about one point support no conclusion. Showing a design can be made still needs your own expression and assays.
Pairs well with: Adaptyv competitions / BenchBB · FoldBench · AIntibody
Sources
- MotifBench paper, arXiv 2502.12479 (preprint) ·
- Paper v2 full text (preprint)
- Official repository and leaderboard (leaderboard) ·
Corrections
2 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| models_reported | RFdiffusion3 and La-Proteina: Chinese model pages state no MotifBench entry was found | both are official-participation entries on the MotifBench board (43.52 and 29.75) | source | 2026-11-16 |
| models_reported | Proteina-Complexa counted as a MotifBench pairing | recorded as not-a-pairing: the method paper never mentions the benchmark | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.