Overview
| Name | Basenji/Enformer test set · aliases: Basenji test set, Basenji2 test split, Enformer test set, cross2020 data |
|---|---|
| Full name | The Basenji/Enformer sequence-to-function test set: the Basenji2 data and its fixed splits |
| Ambiguous names (not auto-linked) | Enformer, Basenji |
| Family | This entry covers the data and its fixed splits, not any model; later models re-fold or change track sets, which counts as their own protocol.
|
| Domain | Genomics & RNA |
| Inclusion | Original 30 |
| Type | B · Static dataset suite with fixed splits and no leaderboard |
| Evidence level | L2–L3 · Main task holds out regions of one genome; the variant task is near L3 |
| Publisher | Calico Life Sciences, DeepMind (Mixed) |
| First release | † partly verified · paper v1 Basis: Earliest family member: the Basenji bioRxiv preprint, journal version 2018-03-27; the fixed splits came later with the Basenji2 data. Note: Family-level date, not the release of the later model of the same name source |
| Other milestones | |
| Latest update | † partly verified · maintenance only Basis: Commit history of the official repository holding the earlier data A dependency-freezing commit in the earlier data repository: code maintenance only, with the benchmark data and splits unchanged. Note: The commit is in the earlier data repository, not in a later model's repository source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | No maintainer has announced a new version or new splits. |
| Status | ● Active · The data are stable and reused by newer models; no leaderboard |
| Scale | 5,313 human tracks and 1,643 mouse tracks; the human test split has 1,937 sequences of 393,216 bp.
|
| Wet-lab refresh | Static data · No new experiments; reuses public epigenomic tracks, with public eQTL and MPRA data for variants. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 3 counted (self 3) |
| Unverified fields | † first_release, latest_update, family.members[1], ranking.entries[1], ranking.entries[2], ranking.entries[0] |
| As of | 2026-11-16 |
What it measures
- Task: Predict thousands of epigenomic and transcriptional tracks from DNA sequence and score variant effects from reference-versus-alternate differences.
- Truth: Public experimental tracks, heterogeneous in batch and coverage; variant labels come from statistical fine-mapping, not experiments.
- Primary metric: Track Pearson correlation at 128 bp bins
- Secondary metrics: Gene-level Pearson correlation; Variant-effect auROC
- Version traps: State which data generation, folds and resolution were used: later models re-fold the data, so per-bin numbers do not compare directly.
Current ranking
Self-reported
| Metric | Track Pearson (128/32/1 bp bins), gene-level Pearson, variant auROC |
|---|---|
| Eval set | Basenji2 held-out test intervals (Borzoi/AlphaGenome use own re-folded splits) |
| Snapshot | |
| Board | No official board |
| Retrieved | 2026-11-16 |
Caveat. There is no leaderboard; each paper uses different track sets, folds and resolutions, so numbers do not compare across papers and relative gains are each paper's own figures.
No headline entry: The sources contain no rank-1 entry and the entries use different protocols, so no headline number is given.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | AlphaGenome (all-folds, pre-trained) | DNase 128bp r=0.87 (+2.4% vs Enformer); CAGE 128bp r=0.71 (-0.3% vs Enformer); RNA-seq 1bp r=0.59 (+28.2% vs Borzoi) | Track Pearson (128/32/1 bp bins), gene-level Pearson, variant auROC Basenji2 held-out test intervals (Borzoi/AlphaGenome use own re-folded splits) | Self-reported | source second-hand Model card checked 2026-11-16; the paper's broader claims of leading were not re-verified |
| — | Enformer | CAGE gene-level Pearson 0.85 (Basenji2 0.81; replicate ceiling 0.94) | Track Pearson (128/32/1 bp bins), gene-level Pearson, variant auROC Basenji2 held-out test intervals (Borzoi/AlphaGenome use own re-folded splits) | Self-reported | source second-hand Not a site model; reference baseline from an existing site note, not re-verified |
| — | Borzoi | gene-level RNA-seq Pearson ~0.87 | Track Pearson (128/32/1 bp bins), gene-level Pearson, variant auROC Basenji2 held-out test intervals (Borzoi/AlphaGenome use own re-folded splits) | Self-reported | source second-hand Second-hand figure from an existing site note; different folds and track set |
| — | NTv3 | No score | Track Pearson (128/32/1 bp bins), gene-level Pearson, variant auROC Basenji2 held-out test intervals (Borzoi/AlphaGenome use own re-folded splits) | Self-reported | source Self-reported on its own task set; no number on this set, so not comparable |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 3. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Developer report, official protocol (3)
- AlphaGenome Self-reported · Self-reported on this data's track test set · evidence: link
- Borzoi Self-reported · The paper uses its own variant tasks and also reports on this test set · evidence: link
- NTv3 Self-reported · Self-report in a technical report, fine-tuned on re-folded splits · evidence: link
Caveats
- Cross-gene correlation mainly reflects which genes are highly expressed, not the ability to predict across individuals.
- An independent study reported a near-zero mean cross-individual correlation, with some genes significantly anticorrelated.
- Gains on reference sequence do not carry over to variant-effect prediction, which is weaker in cell-type-specific regions.
- Zero-shot and fine-tuned results are often mixed, and reuse usually reports a single correlation with no confidence interval.
- Variant-task labels come from statistical fine-mapping, not experimentally validated causal variants.
How to use it
Download the data and fixed splits, predict tracks on held-out regions and score variants from reference-versus-alternate differences; it runs statically with no wet lab of your own. Report the data generation, folds, resolution and track subset, separate zero-shot from fine-tuned results, and give individual-level evaluation separately rather than cross-gene correlation alone. Pair with TraitGym and BEND.
Pairs well with: TraitGym · BEND · GUE + Nucleotide Transformer benchmark · ClinVar hold-out protocol
Sources
- Journal paper establishing the protocol in common use (journal)
- Earliest preprint in the family (preprint)
- Cross-species data paper (journal)
- Repository holding the earlier data and code (repository)
- Third-party study of individual-level performance (third-party paper)
- Later model paper reusing this data (journal)
Corrections
2 corrections
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.