Overview
| Name | FLAb / FLAb2 · aliases: FLAb, FLAb2 |
|---|---|
| Full name | Public materials give no single expanded name. |
| Ambiguous names (not auto-linked) | None |
| Family | Not a family entry. FLAb2 is treated as a new version of the same benchmark, not a separate family member. |
| Domain | Antibodies (also: Proteins) |
| Inclusion | Supplementary entry |
| Type | B · Aggregated public datasets, zero-shot and few-shot settings. |
| Evidence level | L2–L3 · Heterogeneous retrospective data; a few sets carry held-out splits. |
| Publisher | Johns Hopkins University (Academic lab) |
| First release | · paper v1 Basis: bioRxiv preprint v1 (10.1101/2024.01.13.575504) source |
| Other milestones | |
| Latest update | · maintenance only Basis: commits on the GitHub main branch (repository cleanup) Main-branch cleanup: directories removed and README edited; no release since FLAb2 v1.0.0 (2025-12-23). source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | The maintainers have announced no next version or round. |
| Status | ● Active · Repository still receiving commits; FLAb2 is a December 2025 preprint, not peer reviewed. |
| Scale | FLAb2 abstract: 7 properties, 32 studies, over 4M records, 30 models; the README gives different figures.
|
| Wet-lab refresh | Grows from public sources · Community contributes published assay data; the maintainers run no new assays. |
| Own wet lab needed to evaluate a new model | Yes, to claim real generalization |
| Models reported | 1 counted (retrospective 1) |
| Unverified fields | † models_reported[0].source_type, ranking.entries[1].note |
| As of | 2026-11-16 |
What it measures
- Task: Given antibody sequences (sometimes structures), predict developability values: expression, thermostability, aggregation, polyreactivity, binding, immunogenicity, pharmacokinetics.
- Truth: Published antibody assay measurements from public studies, pooled across labs and assays.
- Primary metric: Spearman correlation between model score and measurement.
- Secondary metrics: Few-shot correlation after fine-tuning.
- Version traps: FLAb v1 covers six properties and FLAb2 seven; abstract and README scales differ, so cite version and source.
Current ranking
Third-party
| Metric | Spearman rank correlation (zero-shot and few-shot, per property) |
|---|---|
| Eval set | FLAb2 (241 datasets, 7 therapeutic properties), FLAb2 preprint evaluation |
| Snapshot | |
| Board | No official board |
| Retrieved | 2026-11-16 |
Caveat. No live leaderboard; figures come from the FLAb2 preprint (benchmark authors scoring third-party models, not peer reviewed) and cover one site model on one property.
No headline entry: The source has no rank-1 entry: values belong to different properties and settings, so only the caveat is shown.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| — | IgFold | ρ = 0.12 | Spearman rank correlation (zero-shot and few-shot, per property) FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation | Third-party | source Not a site model; best overall zero-shot result, only 0.12. |
| — | Chai-1 | ρ = 0.45 | Spearman rank correlation (zero-shot and few-shot, per property) FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation | Third-party | source Best zero-shot predictor of thermostability; same paper reports -0.17 for aggregation; no overall value given. |
| — | ESM2 3B and ISM 3B (tied) | ρ = 0.56 | Spearman rank correlation (zero-shot and few-shot, per property) FLAb2 (241 datasets, 7 therapeutic properties); FLAb2 preprint evaluation | Third-party | source Not site models; joint best overall few-shot result. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 1. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Retrospective dataset use (1)
- Chai-1 Third-party zero-shot thermostability eval in the benchmark's own paper · The FLAb2 paper evaluates this model zero-shot on thermostability; two site surfaces label the source type differently. †partly verified · evidence: link
Caveats
- Data are heterogeneous across labs and assays, so one property pools different protocols.
- FLAb2 reports no significant zero-shot correlation on about 80% of datasets; the best overall value is 0.12.
- FLAb2 reports that germline edit distance explains about 40% of language models' apparent skill.
- Scale is inconsistent: abstract and repository README give different dataset and record counts.
- FLAb2 is a preprint, not peer reviewed; uniform-assay blind evidence comes from GDPa1 instead.
How to use it
Take data and scoring scripts from the repository; report zero-shot and few-shot separately, with Spearman correlation per property and the dataset count; first run a germline edit-distance baseline to rule out germline bias; state whether FLAb v1 or FLAb2 is used and any training overlap. Claims about new antibodies need a uniform-assay held-out blind test or own experiments.
Pairs well with: Ginkgo GDPa1 · AbBiBench · AIntibody · ProteinGym · FLIP / FLIP2
Sources
- FLAb2 preprint (bioRxiv, PMC12767642) (preprint) ·
- FLAb v1 preprint (bioRxiv 10.1101/2024.01.13.575504) (preprint) ·
- Official repository Graylab/FLAb (repository)
- FLAb2 release v1.0.0 (repository) ·
Corrections
4 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| latest_update_kind | new-version | code-maintenance-only | source | 2026-11-16 |
| latest_update_desc | FLAb2 models, README and figure merged into Graylab/FLAb main from flab2-dev; fitness unit fixes; legacy dirs removed. Later only README fix 2026-08-17. | Repository cleanup/restructure on main (2026-05-12: envs/analysis/sbatch directories removed, README edited); no release since FLAb2 v1.0.0 (2025-12-23). | source | 2026-11-16 |
| members | [{"name": "FLAb", "first_release": "2024-01-15", "basis": "bioRxiv v1"}, {"name": "FLAb2", "first_release": "2025-12-27", "basis": "bioRxiv v1 10.64898/2025.12.27.696706 (GitHub release v1.0.0 2025-12-23)"}] | [{"name": "FLAb", "first_release": "2024-01-28", "basis": "existing record"}, {"name": "FLAb2", "first_release": "2025-12-23", "basis": "GitHub release v1.0.0, earliest public evidence (precedes bioRxiv v1)"}] | source | 2026-11-16 |
| ranking.headline | headline number shown for FLAb | no headline number; the caveat text is shown instead | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.