Overview
| Name | LAB-Bench / LABBench2 · aliases: LAB-Bench, LABBench2, LitQA2, LitQA3, FigQA, ProtocolQA, SeqQA, SeqQA2, CloningScenarios, CloningQA, DbQA, SuppQA, TableQA |
|---|---|
| Full name | Language Agent Biology Benchmark |
| Ambiguous names (not auto-linked) | LitQA |
| Family | Family entry: original LAB-Bench (multiple choice) and LABBench2 (open answers, tools allowed); question sets and scores are not comparable.
|
| Domain | Text, LLMs & agents |
| Inclusion | Original 30 |
| Type | E · Research-skill question suites; LABBench2 adds executable verifiers |
| Evidence level | L1–L2 · CloningScenarios near L2 |
| Publisher | FutureHouse, Edison Scientific (Mixed) |
| First release | · paper v1 Basis: LAB-Bench arXiv v1 (2407.10362) source |
| Other milestones |
|
| Latest update | · maintenance only Basis: labbench2 repository commit labbench2 harness commit: grading seqqa2 genes from annotated CDS files; several grader and runner fixes in October 2026. source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | No next version or board update announced. |
| Status | ● Active · LABBench2 has an official board (updated 2026-03-23); original subsets partly saturated |
| Scale | Original: 2,457 questions, 30 subtasks; LABBench2: 1,912 tasks, 11 categories, no multiple choice.
|
| Wet-lab refresh | Static data · None; questions written in-house and revised by version. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 3 counted (board 1, modified protocol 2, unconfirmed 5) |
| Unverified fields | † family.members[1].first_release, ranking.entries[5], models_reported[1], models_reported[2] |
| As of | 2026-11-16 |
What it measures
- Task: Biology research skills of LLMs and agents: literature, databases, figures, sequences, cloning and protocol troubleshooting.
- Truth: Expert-written answers, partly programmatic (ProtocolQA, SeqQA); LABBench2 sequence and cloning tasks use executable verifiers.
- Primary metric: Accuracy (LABBench2: verifier or expert grading)
- Secondary metrics: Precision (original); Coverage (original)
- Version traps: Original and LABBench2 differ and cannot share a table; state version, tool use, inline context and board date; vendor system-card subitems are often derived variants.
Current ranking
Board
| Metric | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' |
|---|---|
| Eval set | LABBench2 (1,912 tasks, 11 categories) |
| Snapshot | |
| Board | https://advances.edisonscientific.com/benchmarks/labbench2/ |
| Retrieved | 2026-11-16 |
Caveat. Board lists only 5 models (released 2026-02-05, updated 2026-03-23), no GPT-Rosalind or agent systems. The original has no board; vendor system-card numbers are self-reported subsets.
Shown in the overview table: Board
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| 1 | gpt-5-2 | 0.609 (Tools + high); Base 0.345 | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' LABBench2 (1,912 tasks, 11 categories) | Board | source Overall top-1 (Tools + high); Base top-1 is gemini-3-pro-preview (0.404). |
| 2 | gpt-5-2-pro | 0.583 (Tools + high); Base 0.382 | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' LABBench2 (1,912 tasks, 11 categories) | Board | source |
| 3 | gemini-3-pro-preview | 0.564 (Tools + high); Base 0.404 | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' LABBench2 (1,912 tasks, 11 categories) | Board | source |
| 4 | claude-opus-4-6 | 0.519 (Tools + high); Base 0.349 | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' LABBench2 (1,912 tasks, 11 categories) | Board | source |
| 5 | claude-opus-4-5 | 0.471 (Tools + high); Base 0.325 | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' LABBench2 (1,912 tasks, 11 categories) | Board | source |
| — | Biomni | Withheld: Incomplete provenance (missing url and date): the honesty rule forbids displaying this number. | LABBench2 aggregate score (mean across sub-benchmarks), settings 'Tools + high' and 'Base' Original LAB-Bench DbQA/SeqQA subsets (not LABBench2) | Self-reported No sourced date† | no source link second-hand Original LAB-Bench subsets, not LABBench2; from site note (second-hand), not re-verified; not comparable to the board. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 3. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Official leaderboard or results (1)
- Frontier general-purpose LLMs Board LABBench2 · LABBench2 official board entries; family entry. · evidence: link
Developer report, modified protocol (2)
- Biomni Self-reported LAB-Bench DbQA/SeqQA subsets · Original DbQA/SeqQA subsets with 45 dev / 315 hidden test questions and 3 runs. low confidence †unverified · evidence: link
- GPT-Rosalind Self-reported · 11 of 15 LABBench2 tasks, qualitative only. †partly verified · evidence: link
Unconfirmed (5, not counted)
- Co-Scientist Unconfirmed source · No report found in official materials †disputed · evidence: link
- Intern-S Unconfirmed source · No report found in official materials †disputed · evidence: link
- Quine Unconfirmed source · No report found in official materials †disputed · evidence: link
- SciReasoner Unconfirmed source · Not in the paper text †disputed · evidence: no source link
- VERID Unconfirmed source · No report found in official materials †disputed · evidence: no source link
Caveats
- Multiple choice inflated original scores; small subsets (41 and 135 questions) are noisy; the public 80% may be contaminated; no wet-lab evidence.
- The LABBench2 paper reports accuracy 26–46 points below the original; original scores do not represent open-answer ability.
- Vendor system-card subitems (e.g. open-ended ProtocolQA) are derived variants, and the same subitem circulates with several numbers; use the primary document.
- ProtocolQA is paper-based error finding and involves no actual experiment.
How to use it
Run from the repository or Hugging Face data; no wet lab needed; the original needs an agent function returning the option letter. Prefer LABBench2 and state tool use (search, code execution), inline context and board date; the private 20% hold-out cannot be checked for contamination. Pair with BixBench and wet-lab blind tests such as Adaptyv.
Pairs well with: BixBench · Adaptyv competitions / BenchBB · PubMedQA · BioASQ
Sources
- LAB-Bench paper, arXiv 2407.10362 (preprint)
- LABBench2 paper, arXiv 2604.09554 (preprint)
- LABBench2 official board (leaderboard)
- LABBench2 release page (company blog)
- labbench2 repository (repository)
- LAB-Bench repository (repository)
- LAB-Bench on Hugging Face (repository)
- Claude Opus 4.5 system card (vendor self-report) (company blog)
- Biomni preprint (developer) (preprint)
Corrections
1 correction
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| ranking.entries[Biomni] | Biomni original LAB-Bench subset scores shown as a ranking entry | Score withheld: entry lacks url and date | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.