AI for Bio · Benchmark 34

AbBiBench

Zero-shot scoring correlations evaluated on antibody-antigen complexes, filling a gap in antibody function evaluation.

bench-34Type BEvidence L2–L3Antibodies● Active Board中文

Overview

NameAbBiBench · aliases: AbBiBench, Antibody Binding Benchmark Dataset
Full nameAbBiBench: antibody-antigen binding benchmark
Ambiguous names (not auto-linked)None
FamilyNot a family entry. One dataset with one maintainer board; no family members.
DomainAntibodies
InclusionSupplementary entry
TypeB · Fixed set with a zero-shot correlation board
Evidence levelL2–L3 · Retrospective published measurements, evaluated zero-shot
PublisherAcademic biomedical informatics group (repository MSBMI-SAFE) (Academic lab)
First release · paper v1
Basis: arXiv 2506.04235 v1 (2025-05-23); the Hugging Face dataset and repository creation dates are not used as first release source
Other milestones
  • First public evidence: † · Paper v1 is the earliest verifiable public evidence source
  • Preprint v1: · arXiv 2506.04235 v1 source
Latest update · paper revision
Basis: arXiv paper v3 revision
Paper revised to arXiv v3; the repository board was aligned to the manuscript evaluation on 2026-10-19. source
The most recent sourced event, not a claim that nothing changed since.
Next expectedThe maintainers have announced no new version or evaluation round.
Status● Active · Active; the repository board is a snapshot matching the paper
Scale216,560 measurements, 13 antibodies, 9 antigens (v3 abstract); Hugging Face copy about 215,699 rows.
  • Measurements: 216,560 measurements (arXiv v3)
  • Antibodies / antigens: 13 / 9 (arXiv v3)
  • Assays: 16 assays (board)
  • Base methods / implementations: 20 / 23 (board)
  • Hugging Face dataset rows: ~215,699 rows (Hugging Face copy)
  • Design validation candidates: 21 candidates (arXiv v3)
Wet-lab refreshStatic data · Reuses published binding measurements; the only new experiment tests 21 candidates for reactivity.
Own wet lab needed to evaluate a new modelNo, run it as a download
Models reported6 counted (board 6)
Unverified fields† release_milestones.first_public_evidence, ranking.snapshot_date, ranking.entries[0], ranking.entries[1], ranking.entries[2], ranking.entries[3], ranking.entries[4], ranking.entries[5], ranking.entries[6], models_reported[0], models_reported[1], models_reported[2], models_reported[3], models_reported[4], models_reported[5]
As of2026-11-16

What it measures

  • Task: Zero-shot correlation between model scores on antibody-antigen complexes and measured binding, plus one small design validation.
  • Truth: Published binding measurements kept separate by assay, with different units never mixed.
  • Primary metric: Per-assay correlation between score and measurement
  • Secondary metrics: Approximate mean correlation over 16 assays; Reactivity signal for design candidates
  • Version traps: Older circulating size figures are superseded by v3; the board header and the page note describe the statistic differently, and the page note governs.

Current ranking

Board

MetricApprox. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
Eval setAbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Snapshot
Boardhttps://github.com/MSBMI-SAFE/AbBiBench
Retrieved2026-11-16

Caveat. The board sits in the repository README and states it is a manuscript-matched snapshot: values are copied from a paper figure, not an official rerun; some runners and per-variant results are not public, and rounded means can tie.

Shown in the overview table:

ProteinMPNN
0.29
Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately · AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board ·
Best site model: Boltz-2 (ipSAE) (#4)

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
1ProteinMPNN0.29Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Not a site model; overall first; an inverse-folding method.
4Boltz-2 (ipSAE)0.14Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Structure-prediction category; the highest-ranked site model.
10ESM-30.05Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Masked language model category.
11SaProt0.04Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Masked language model category.
12ESMFold2 (ipSAE)0.04Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Structure-prediction category.
16ProSST-0.00Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Masked language model category.
19AF3 (ipSAE)-0.03Approx. mean rho (header reads 'Approx. mean ρ² ↑'; page states values are two-decimal Spearman correlations from manuscript Figure 3a), each of 16 assays evaluated separately
AbBiBench manuscript-matched snapshot, 20 base methods / 23 implementations across 16 assays
Board
source Structure-prediction category.

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 6. A pairing whose evidence rests only on this survey's own page is shown as unverified.

Official leaderboard or results (6)

  • AlphaFold 3 Board registered interface-score entry · Benchmark-registered ipSAE correlation (not structure accuracy), read from paper figure, not rerun †unverified · evidence: no source link
  • Boltz-2 Board registered interface-score entry · Benchmark-registered ipSAE entry; not structure accuracy †unverified · evidence: no source link
  • ESM3 Board benchmark-registered entry · Benchmark-registered score †unverified · evidence: no source link
  • ESMFold2 Board AbBiBench structure-prediction entry · Benchmark-registered ipSAE entry †unverified · evidence: no source link
  • ProSST Board antibody binding-affinity correlation, structure-aware LM category · Benchmark-registered score †unverified · evidence: no source link
  • SaProt Board antibody binding-affinity correlation, structure-aware LM category · Benchmark-registered score †unverified · evidence: no source link

Caveats

  1. Correlations are low (the top method averages about 0.29 across 16 assays); assays must be reported separately and units never mixed.
  2. The design validation covers 21 candidates with no reported controls, replicates or thresholds, so it shows signal only.
  3. Board values are copied from a paper figure rather than an official rerun; some runners and per-variant results are unpublished, limiting replication.
  4. The paper states the ranking suits prioritising promising variants rather than excluding weak ones.
  5. The data are reused published measurements shaped by the original studies rather than one uniform assay campaign.

How to use it

Score the public complex set zero-shot with the repository scripts and report per-assay correlations rather than a pooled mean; state method category, score definition, any antibody-specific fine-tuning and the structure source. Claims of replacing experimental triage need prospective measurements.

Pairs well with: AIntibody · FLAb / FLAb2 · FoldBench · Adaptyv competitions / BenchBB

Sources

  1. AbBiBench paper (arXiv 2506.04235) (preprint)
  2. AbBiBench official repository and board (leaderboard)
  3. AbBiBench dataset (Hugging Face, CC-BY-4.0) (repository)

Corrections

3 corrections
FieldWasNowBasisFound
scaleabout 184,500 measurements, 14 antibodies, 15 models216,560 measurements, 13 antibodies, 9 antigens (arXiv v3); 20 base methods / 23 implementations on the boardsource2026-11-16
wetlab_noteone wet-lab round of 21 candidates described as affinity validationthe 216,560 values are reused published measurements and the only new experiment is a 21-candidate reactivity test, not an affinity measurementsource2026-11-16
models_reportedsix entries labelled as board resultsboard values are correlations copied from a paper figure, so the entries are flagged unverifiedsource2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.