AI for Bio · Benchmark 35

FLAb / FLAb2

Aggregated retrospective antibody developability benchmark with a negative headline result.

bench-35Type BEvidence L2–L3Antibodies● Active Third-party中文

Overview

NameFLAb / FLAb2 · aliases: FLAb, FLAb2
Full namePublic materials give no single expanded name.
Ambiguous names (not auto-linked)None
FamilyNot a family entry. FLAb2 is treated as a new version of the same benchmark, not a separate family member.
DomainAntibodies (also: Proteins)
InclusionSupplementary entry
TypeB · Aggregated public datasets, zero-shot and few-shot settings.
Evidence levelL2–L3 · Heterogeneous retrospective data; a few sets carry held-out splits.
PublisherJohns Hopkins University (Academic lab)
First release · paper v1
Basis: bioRxiv preprint v1 (10.1101/2024.01.13.575504) source
Other milestones
  • First public evidence: No sourced date† · Repository creation is excluded as first-release evidence, so no date is recorded. source
  • Preprint v1: · bioRxiv preprint v1 of FLAb v1 source
  • Code release: · GitHub release v1.0.0 for FLAb2 source
Latest update · maintenance only
Basis: commits on the GitHub main branch (repository cleanup)
Main-branch cleanup: directories removed and README edited; no release since FLAb2 v1.0.0 (2025-12-23). source
The most recent sourced event, not a claim that nothing changed since.
Next expectedThe maintainers have announced no next version or round.
Status● Active · Repository still receiving commits; FLAb2 is a December 2025 preprint, not peer reviewed.
ScaleFLAb2 abstract: 7 properties, 32 studies, over 4M records, 30 models; the README gives different figures.
  • Property classes: 7 (FLAb2)
  • Studies included: 32 (abstract)
  • Antibody records: >4,000,000 (abstract)
  • Models evaluated: 30 (abstract)
  • Datasets: 241 (repository README)
  • Assay records: >3,000,000 (repository README)
Wet-lab refreshGrows from public sources · Community contributes published assay data; the maintainers run no new assays.
Own wet lab needed to evaluate a new modelYes, to claim real generalization
Models reported1 counted (retrospective 1)
Unverified fields† models_reported[0].source_type, ranking.entries[1].note
As of2026-11-16

What it measures

  • Task: Given antibody sequences (sometimes structures), predict developability values: expression, thermostability, aggregation, polyreactivity, binding, immunogenicity, pharmacokinetics.
  • Truth: Published antibody assay measurements from public studies, pooled across labs and assays.
  • Primary metric: Spearman correlation between model score and measurement.
  • Secondary metrics: Few-shot correlation after fine-tuning.
  • Version traps: FLAb v1 covers six properties and FLAb2 seven; abstract and README scales differ, so cite version and source.

Current ranking

Third-party

MetricSpearman rank correlation (zero-shot and few-shot, per property)
Eval setFLAb2 (241 datasets, 7 therapeutic properties), FLAb2 preprint evaluation
Snapshot
BoardNo official board
Retrieved2026-11-16

Caveat. No live leaderboard; figures come from the FLAb2 preprint (benchmark authors scoring third-party models, not peer reviewed) and cover one site model on one property.

No headline entry: The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is shown.

RankModelScore (verbatim)Metric · eval set / protocolSource type · dateSource and note
—IgFoldρ = 0.12Spearman rank correlation (zero-shot and few-shot, per property)
FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation
Third-party
source Not a site model; best overall zero-shot result, only 0.12.
—Chai-1ρ = 0.45Spearman rank correlation (zero-shot and few-shot, per property)
FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation
Third-party
source Best zero-shot predictor of thermostability; same paper reports -0.17 for aggregation; no overall value given.
—ESM2 3B and ISM 3B (tied)ρ = 0.56Spearman rank correlation (zero-shot and few-shot, per property)
FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation
Third-party
source Not site models; joint best overall few-shot result.

Models reported

Grouped by relation type; each line shows its source type. Counted relations add up to 1. A pairing whose evidence rests only on this survey's own page is shown as unverified.

Retrospective dataset use (1)

  • Chai-1 Third-party zero-shot thermostability eval in the benchmark's own paper · The FLAb2 paper evaluates this model zero-shot on thermostability; two site surfaces label the source type differently. †partly verified · evidence: link

Caveats

  1. Data are heterogeneous across labs and assays, so one property pools different protocols.
  2. FLAb2 reports no significant zero-shot correlation on about 80% of datasets; the best overall value is 0.12.
  3. FLAb2 reports that germline edit distance explains about 40% of language models' apparent skill.
  4. Scale is inconsistent: abstract and repository README give different dataset and record counts.
  5. FLAb2 is a preprint, not peer reviewed; uniform-assay blind evidence comes from GDPa1 instead.

How to use it

Take data and scoring scripts from the repository; report zero-shot and few-shot separately, with Spearman correlation per property and the dataset count; first run a germline edit-distance baseline to rule out germline bias; state whether FLAb v1 or FLAb2 is used and any training overlap. Claims about new antibodies need a uniform-assay held-out blind test or own experiments.

Pairs well with: Ginkgo GDPa1 · AbBiBench · AIntibody · ProteinGym · FLIP / FLIP2

Sources

  1. FLAb2 preprint (bioRxiv, PMC12767642) (preprint) ·
  2. FLAb v1 preprint (bioRxiv 10.1101/2024.01.13.575504) (preprint) ·
  3. Official repository Graylab/FLAb (repository)
  4. FLAb2 release v1.0.0 (repository) ·

Corrections

4 corrections
FieldWasNowBasisFound
latest_update_kindnew-versioncode-maintenance-onlysource2026-11-16
latest_update_descFLAb2 models, README and figure merged into Graylab/FLAb main from flab2-dev; fitness unit fixes; legacy dirs removed. Later only README fix 2026-08-17.Repository cleanup/restructure on main (2026-05-12: envs/analysis/sbatch directories removed, README edited); no release since FLAb2 v1.0.0 (2025-12-23).source2026-11-16
members[{"name": "FLAb", "first_release": "2024-01-15", "basis": "bioRxiv v1"}, {"name": "FLAb2", "first_release": "2025-12-27", "basis": "bioRxiv v1 10.64898/2025.12.27.696706 (GitHub release v1.0.0 2025-12-23)"}][{"name": "FLAb", "first_release": "2024-01-28", "basis": "existing record"}, {"name": "FLAb2", "first_release": "2025-12-23", "basis": "GitHub release v1.0.0, earliest public evidence (precedes bioRxiv v1)"}]source2026-11-16
ranking.headlineheadline number shown for FLAbno headline number; the caveat text is shown insteadsource2026-11-16

Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.