Overview
| Name | AbBiBench · aliases: AbBiBench, Antibody Binding Benchmark Dataset |
|---|---|
| Full name | AbBiBench: antibody-antigen binding benchmark |
| Ambiguous names (not auto-linked) | None |
| Family | Not a family entry. One dataset with one maintainer board; no family members. |
| Domain | Antibodies |
| Inclusion | Supplementary entry |
| Type | B · Fixed set with a zero-shot correlation board |
| Evidence level | L2–L3 · Retrospective published measurements, evaluated zero-shot |
| Publisher | Academic biomedical informatics group (repository MSBMI-SAFE) (Academic lab) |
| First release | · paper v1 Basis: arXiv 2506.04235 v1 (2025-05-23); the Hugging Face dataset and repository creation dates are not used as first release source |
| Other milestones | |
| Latest update | · paper revision Basis: arXiv paper v3 revision Paper revised to arXiv v3; the repository board was aligned to the manuscript evaluation on 2026-10-19. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | The maintainers have announced no new version or evaluation round. |
| Status | ● Active · Active; the repository board is a snapshot matching the paper |
| Scale | 216,560 measurements, 13 antibodies, 9 antigens (v3 abstract); Hugging Face copy about 215,699 rows.
|
| Wet-lab refresh | Static data · Reuses published binding measurements; the only new experiment tests 21 candidates for reactivity. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 6 counted (board 6) |
| Unverified fields | † release_milestones.first_public_evidence, ranking.snapshot_date, ranking.entries[0], ranking.entries[1], ranking.entries[2], ranking.entries[3], ranking.entries[4], ranking.entries[5], ranking.entries[6], models_reported[0], models_reported[1], models_reported[2], models_reported[3], models_reported[4], models_reported[5] |
| As of | 2026-11-16 |
What it measures
- Task: Zero-shot correlation between model scores on antibody-antigen complexes and measured binding, plus one small design validation.
- Truth: Published binding measurements kept separate by assay, with different units never mixed.
- Primary metric: Per-assay correlation between score and measurement
- Secondary metrics: Approximate mean correlation over 16 assays; Reactivity signal for design candidates
- Version traps: Older circulating size figures are superseded by v3; the board header and the page note describe the statistic differently, and the page note governs.
Current ranking
Board
| Metric | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately |
|---|---|
| Eval set | AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays |
| Snapshot | |
| Board | https://github.com/MSBMI-SAFE/AbBiBench |
| Retrieved | 2026-11-16 |
Caveat. The board sits in the repository README and states it is a manuscript-matched snapshot: values are copied from a paper figure, not an official rerun; some runners and per-variant results are not public, and rounded means can tie.
Shown in the overview table: Board
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| 1 | ProteinMPNN | 0.29 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Not a site model; overall first; an inverse-folding method. |
| 4 | Boltz-2 (ipSAE) | 0.14 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Structure-prediction category; the highest-ranked site model. |
| 10 | ESM-3 | 0.05 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Masked language model category. |
| 11 | SaProt | 0.04 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Masked language model category. |
| 12 | ESMFold2 (ipSAE) | 0.04 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Structure-prediction category. |
| 16 | ProSST | -0.00 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Masked language model category. |
| 19 | AF3 (ipSAE) | -0.03 | Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays | Board | source Structure-prediction category. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 6. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Official leaderboard or results (6)
- AlphaFold 3 Board registered interface-score entry · Benchmark-registered ipSAE correlation (not structure accuracy), read from paper figure, not rerun †unverified · evidence: no source link
- Boltz-2 Board registered interface-score entry · Benchmark-registered ipSAE entry; not structure accuracy †unverified · evidence: no source link
- ESM3 Board benchmark-registered entry · Benchmark-registered score †unverified · evidence: no source link
- ESMFold2 Board AbBiBench structure-prediction entry · Benchmark-registered ipSAE entry †unverified · evidence: no source link
- ProSST Board antibody binding-affinity correlation, structure-aware LM category · Benchmark-registered score †unverified · evidence: no source link
- SaProt Board antibody binding-affinity correlation, structure-aware LM category · Benchmark-registered score †unverified · evidence: no source link
Caveats
- Correlations are low (the top method averages about 0.29 across 16 assays); assays must be reported separately and units never mixed.
- The design validation covers 21 candidates with no reported controls, replicates or thresholds, so it shows signal only.
- Board values are copied from a paper figure rather than an official rerun; some runners and per-variant results are unpublished, limiting replication.
- The paper states the ranking suits prioritising promising variants rather than excluding weak ones.
- The data are reused published measurements shaped by the original studies rather than one uniform assay campaign.
How to use it
Score the public complex set zero-shot with the repository scripts and report per-assay correlations rather than a pooled mean; state method category, score definition, any antibody-specific fine-tuning and the structure source. Claims of replacing experimental triage need prospective measurements.
Pairs well with: AIntibody · FLAb / FLAb2 · FoldBench · Adaptyv competitions / BenchBB
Sources
- AbBiBench paper (arXiv 2506.04235) (preprint)
- AbBiBench official repository and board (leaderboard)
- AbBiBench dataset (Hugging Face, CC-BY-4.0) (repository)
Corrections
3 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| scale | about 184,500 measurements, 14 antibodies, 15 models | 216,560 measurements, 13 antibodies, 9 antigens (arXiv v3); 20 base methods / 23 implementations on the board | source | 2026-11-16 |
| wetlab_note | one wet-lab round of 21 candidates described as affinity validation | the 216,560 values are reused published measurements and the only new experiment is a 21-candidate reactivity test, not an affinity measurement | source | 2026-11-16 |
| models_reported | six entries labelled as board results | board values are correlations copied from a paper figure, so the entries are flagged unverified | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.