AI for Bio · Benchmark 25

RNA-Puzzles

Community blind test of RNA 3D structure; accuracy tracks template availability more than model type.

bench-25Type A+BEvidence L2–L4Structure & co-folding◐ Low-frequency Board中文

Overview

NameRNA-Puzzles · aliases: RNA-Puzzles, RNA-Puzzles Round V, Stanford RNA 3D Folding
Full nameRNA-Puzzles: community blind assessment of RNA 3D structure prediction
Ambiguous names (not auto-linked)Ribonanza, bpRNA, ArchiveII
FamilyNot a family entry. RNA-Puzzles rounds are editions of one blind test; the Kaggle and CASP RNA tracks are related evaluations recorded in scale and notes, not family members.
DomainStructure & co-folding
InclusionOriginal 30
TypeA B · Periodic blind test plus static secondary-structure sets
Evidence levelL2–L4 · 3D blind test L4; static 2D sets L2
PublisherRNA-Puzzles consortium (academic groups), Stanford University (Kaggle RNA 3D Folding) (Academic lab)
First release
Basis: RNA-Puzzles founding paper (RNA 2012, PubMed epub 2012-02-23); puzzles began around 2010-11 source
Other milestones
  • Journal online: · Founding paper online (RNA 2012) source
  • Challenge opens: † · First puzzles began running (per founding paper and site notes) source
Latest update† unverified · new round
Basis: Official-site item "PZ74-PZ79 update: DNA residue names", date derived from its URL only
Official-site PZ74-PZ79 DNA residue-name update, date URL-derived and unverified; last formal round is Round V (paper online 2024-12-02).
Note: Date not reproduced on re-check; the last round, Round V, went online 2024-12-02 source
The most recent sourced event, not a claim that nothing changed since.
Next expectedNo Round VI announced as of 2026-11-16.
Status◐ Low-frequency · Round V published 2025; CASP RNA and Kaggle rounds continue
ScaleRound V: 23 puzzles, 18 groups; CASP16: 42 targets; Kaggle: 20 private targets.
  • Puzzles: 23 puzzles (Round V)
  • Participating groups: 18 groups (Round V)
  • CASP15 RNA targets / groups: 12 / 42 (CASP15)
  • CASP16 nucleic-acid targets / groups: 42 / 65 (CASP16)
  • Kaggle private targets (10 from RNA-Puzzles): 20 targets (Kaggle RNA 3D Folding Part 1)
  • Kaggle teams: 1,700+ teams (Kaggle RNA 3D Folding Part 1)
Wet-lab refreshFresh experiments each round · Round-based, 12-42 targets per round from pre-publication experimental structures; new truth only by entering the next round.
Own wet lab needed to evaluate a new modelNo, run it as a download
Models reported3 counted (modified protocol 2, retrospective 1, cited only 1)
Unverified fields† latest_update, release_milestones.challenge_open, models_reported[1], models_reported[2], models_reported[3]
As of2026-11-16

What it measures

  • Task: RNA 3D structure prediction from sequence, with related static sets for secondary structure and chemical reactivity.
  • Truth: X-ray, cryo-EM or NMR structures unpublished before assessment.
  • Primary metric: RMSD, INF and DI (RNA-Puzzles)
  • Secondary metrics: TM-score (US-align, C1'); GDT_TS, lDDT (CASP and Kaggle)
  • Version traps: RNA-Puzzles, CASP RNA and Kaggle differ in targets, metrics and rules; human groups may use literature and constraints, unlike servers.

Current ranking

Board

MetricMean TM-align (US-align, C1')
Eval setKaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Snapshot
Boardhttps://www.kaggle.com/competitions/stanford-rna-3d-folding
Retrieved2026-11-16

Caveat. Numbers from Table 2 of the competition preprint, reproducing the Kaggle Part 1 private board; Part 2 standings unknown; RNA-Puzzles itself has no standing board.

Shown in the overview table:

Kaggle private-board rank 1 team
0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10)
Mean TM-align (US-align, C1') · Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board ·
Best site model: AlphaFold 3

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
1Kaggle private-board rank 1 team0.578 (all, n=20); 0.491 (RNA-Puzzles, n=10)Mean TM-align (US-align, C1')
Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board
source Not a site model; overall first; a template-discovery pipeline without deep learning.
2Kaggle private-board rank 2 team0.562 (all, n=20); 0.418 (RNA-Puzzles, n=10)Mean TM-align (US-align, C1')
Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Board
source Not a site model.
—AlphaFold 30.464 (all, n=20); 0.396 (RNA-Puzzles, n=10)Mean TM-align (US-align, C1')
Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Third-party
source Reference comparison row in Table 2, not a ranked competition entry.
—trRosettaRNA server0.438 (all, n=20); 0.340 (RNA-Puzzles, n=10)Mean TM-align (US-align, C1')
Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Third-party
source Reference comparison row in Table 2, not a ranked competition entry.
—RNAPro0.648 (all, n=20); 0.541 (RNA-Puzzles, n=10)Mean TM-align (US-align, C1')
Kaggle Stanford RNA 3D Folding Part 1 private board: 20 targets incl. 10 RNA-Puzzles
Self-reported
source Post-competition retrospective model, not a board entry; needs prospective validation.
—NeuralPLexer3mean RNA lDDT 46.5 (AF3 47.3, AIchemy_RNA2 54.8)Mean RNA lDDT
CASP15 RNA, 8-target subset used in the AlphaFold 3 paper
Self-reported
source CASP15 RNA 8-target subset with a different protocol and metric; not comparable with the Kaggle TM-align column.

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 3. A pairing whose evidence rests only on this survey's own page is shown as unverified.

Developer report, modified protocol (2)

  • NeuralPLexer3 Self-reported · CASP15 RNA 8-target subset; AF3 baseline copied from its paper, not rerun · evidence: link
  • RNAPro Self-reported RNA-Puzzles subset within a competition private leaderboard · 10-target RNA-Puzzles subset inside Kaggle private leaderboard †partly verified · evidence: link

Retrospective dataset use (1)

Cited as comparison only (1, not counted)

Caveats

  1. Accuracy depends on available templates rather than model type; the 2025 Kaggle winner was a non-deep-learning template pipeline; secondary-structure sets leak same-family sequences.
  2. Very few targets (at most 23 per round, 20 on the Kaggle private board), with no confidence intervals.
  3. The CASP16 assessment reports no significant progress once templates are controlled, almost no success on template-free targets; RNA-protein complexes still fail.
  4. TM-score uses C1' atoms only and ignores non-canonical interactions.
  5. The latest-update date is URL-derived and unverified; Kaggle Part 2 results were not retrieved.

How to use it

Run released targets statically with the RNA_assessment scripts and US-align; true blind tests require the next CASP or Kaggle round. Stratify by template availability, add a nearest-PDB-template baseline, state target set and models per target; do not claim 2D generalization from random splits.

Pairs well with: CASP · RNAGym · FoldBench

Sources

  1. RNA-Puzzles founding paper (RNA 2012) (journal)
  2. RNA-Puzzles Round V (Nat Methods 2025) (journal)
  3. RNA-Puzzles official site (official site)
  4. Kaggle Stanford RNA 3D Folding competition paper (preprint) (preprint)
  5. Kaggle RNA 3D Folding Part 1 (leaderboard)
  6. Kaggle RNA 3D Folding Part 2 (leaderboard)
  7. CASP15 RNA assessment (third-party paper)
  8. CASP16 nucleic-acid assessment (third-party paper)
  9. RNA_assessment scripts (repository)

Corrections

2 corrections
FieldWasNowBasisFound
latest_update_descKaggle Stanford RNA 3D Folding Part 2 final submission deadline; RNA-Puzzles proper last round V paper (online 2024-12-02). No Round VI announced.Site item 'PZ74-PZ79 update: DNA residue names' dated 2026-04-02 from its URL only; the official site did not return this date or a dated news entry on re-check, so the date must display as unverified. Last RNA-Puzzles round proper: Round V paper online 2024-12-02. The 2026-03-25 Kaggle deadline belongs to the Kaggle component, not RNA-Puzzles.source2026-11-16
latest_update2026-03-252026-04-02source2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.