Overview
| Name | CAFA · aliases: CAFA, CAFA1, CAFA2, CAFA3, CAFA4, CAFA5, CAFA6, Critical Assessment of Functional Annotation |
|---|---|
| Full name | Critical Assessment of Functional Annotation |
| Ambiguous names (not auto-linked) | None |
| Family | Not a family entry. Single recurring challenge; CAFA1–6 are editions of one challenge, not family members. |
| Domain | Proteins |
| Inclusion | Original 30 |
| Type | A · Time-delayed recurring blind challenge |
| Evidence level | L4–L5 · CAFA3 added dedicated screens (L5) |
| Publisher | CAFA organisers (ISCB Automated Function Prediction SIG) (Academic lab) |
| First release | † partly verified Basis: challenge_open: CAFA1 targets released 2010-09-15 (organisers' paper, p. 9). Registration opened 2010-07-15, stored separately and not used as first release. Note: Day 2010-09-15 from the organisers' paper; kept at month source |
| Other milestones |
|
| Latest update | † unverified · new round Basis: Organisers' mailing-list announcement CAFA6 launched on Kaggle ($50k prizes) per the organisers' mailing-list post; final results not found. Note: No official dated record; shown as unverified source The most recent sourced event, not a claim that nothing changed since. |
| Next expected | CAFA6 final evaluation timing (about 2026-05) appears only on aggregator sites; nothing announced by the organisers. |
| Status | ● Active · CAFA6 opened on Kaggle 2025-10-16; no official results as of 2026-11-16 |
| Scale | CAFA2: 126 methods, 3,681 proteins; CAFA5: 1,600+ teams; CAFA6 evaluation-set size not published.
|
| Wet-lab refresh | Grows from public sources · Every 2–3 years; new truth is experimental GO annotation accumulated in databases, not new experiments, and comes only by entering the next edition. |
| Own wet lab needed to evaluate a new model | No, run it as a download |
| Models reported | 0 counted (unconfirmed 1) |
| Unverified fields | † first_release, release_milestones.challenge_open, release_milestones.first_public_evidence, release_milestones.journal_online, latest_update, ranking.entries[0], models_reported[0] |
| As of | 2026-11-16 |
What it measures
- Task: Time-delayed prediction of protein function as Gene Ontology terms, scored against experimental annotations added after submission.
- Truth: Experimental-evidence GO annotations added to UniProt/GOA between t0 and t1, propagated up the GO hierarchy.
- Primary metric: Fmax
- Secondary metrics: Smin (IA-weighted); IA-weighted F-measure (CAFA5/6 Kaggle)
- Version traps: CAFA1–4 used academic submissions with independent assessment, CAFA5/6 are Kaggle contests with different metrics, so scores are not comparable across editions; separate no-knowledge and limited-knowledge.
Current ranking
No ranking
| Metric | IA-weighted F-measure (CAFA-evaluator, Kaggle) |
|---|---|
| Eval set | CAFA6 (Kaggle, launched 2025-10-16) |
| Snapshot | |
| Board | https://www.kaggle.com/competitions/cafa-6-protein-function-prediction/leaderboard |
| Retrieved | 2026-11-16 |
Caveat. CAFA6 final board and winners not retrievable; no site model reports CAFA (SciReasoner pairing unconfirmed); CAFA5 scores are not comparable.
No headline entry: CAFA6 final board and winners were not found; the only entry is an interim snapshot, so there is no headline number.
| Rank | Model | Score (verbatim) | Metric · eval set / protocol | Source type · date | Source and note |
|---|---|---|---|---|---|
| 1 | (individual Kaggle participant; interim) | 0.36 | IA-weighted F-measure (CAFA-evaluator, Kaggle) CAFA6 (Kaggle, launched 2025-10-16) | Board | source Not a site model; interim public-board snapshot via a third-party aggregator with 57 days left, not final; participant name omitted. |
Models reported
Grouped by relation type; each line shows its source type. Counted relations add up to 0. A pairing whose evidence rests only on this survey's own page is shown as unverified.
Unconfirmed (1, not counted)
- SciReasoner Unconfirmed source · Self-reports on DeepFRI-GO; CAFA-3 only in a figure caption, a modified protocol if kept; paper identity unresolved (two same-name papers). †disputed · evidence: link
Caveats
- Unannotated terms count as negatives (open-world problem); CAFA5/6 moved to Kaggle, so scores are not comparable across editions; text-mining entries are allowed.
- Fmax favours shallow, frequent terms, and rankings shift substantially with the metric.
- Top CAFA5 teams used text mining of new papers as features, which is legal in time but changes what the task measures.
- Top-method gains from CAFA2 to CAFA3 were smaller than the previous step and CCO did not improve; the bottleneck is annotation, not models.
- CAFA3 included prospective experiments, but the mapping from experimental hit rate to Fmax is not quantified.
How to use it
Enter the next edition, or build a time split (train on an old snapshot, evaluate on a newer one) and compute Fmax and Smin with CAFA-evaluator, next to Naive and BLAST baselines, reporting no-knowledge and limited-knowledge and IA-weighted versions. No own wet lab needed, but scores reflect only documented function.
Pairs well with: CASP · CAGI · ProteinGym
Sources
- CAFA6 on Kaggle (official site)
- CAFA6 organisers' mailing-list announcement (official site)
- CAFA official site (official site)
- CAFA1 results paper (Nature Methods 2013) (journal)
- CAFA2 paper (PubMed) (journal)
- CAFA3 paper (Genome Biology 2019) (journal)
- Archived organisers' paper (CAFA1 dates) (archive)
- ProtBoost (CAFA5 team paper) (third-party paper)
- CAFA6 interim standings aggregator (second-hand) (second-hand) second-hand
Corrections
3 corrections
| Field | Was | Now | Basis | Found |
|---|---|---|---|---|
| first_release | 2010-09 (challenge began 2010-09-15; registration 2010-07-15) | 2010-09 | source | 2026-11-16 |
| first_release_basis | CAFA1 organized fall 2010 to spring 2012 per secondary summaries; month not found | challenge_open: CAFA1 targets released 2010-09-15 (organisers' paper, p. 9). Registration opened 2010-07-15, stored separately and not used as first release. | source | 2026-11-16 |
| first_release | 2010-fall | 2010-09 (challenge began 2010-09-15; registration 2010-07-15) | source | 2026-11-16 |
Part of the AI for Bio benchmarks survey · all 44 benchmarks · open in the explorer. Record as of 2026-11-16.