AI for Bio · Model

MoLFormer-XL

Chemical language model pretraining on billion-scale molecule strings, then fine-tuned for property prediction

Small moleculesBig-tech labUnited StatesMaintained only中文

Overview

NameMoLFormer-XL · aliases: MoLFormer-XL, MoLFormer-XL-both-10pct
Full nameThe paper gives no expanded name
Ambiguous names (not auto-linked)MoLFormer
FamilyNot a family entry. The later generative variant from the same organisation has no site entity record, so no family entry
OrganizationIBM Research
Organization typeBig-tech lab
CountryUnited States
Model classProperty prediction (molecule-string language model)
DomainSmall molecules
First release · paper revision
Basis: arXiv 2106.09553 v1 submitted
Note: The site used the 2022 journal year as the release; first release follows the preprint v1 source
Other milestones
  • First public evidence: · arXiv v1 source
  • Preprint v1: · arXiv v1 (figures follow the v3 full text) source
  • Journal online: · Journal publication (DOI 10.1038/s42256-022-00580-7) source
  • Code release: c. 2022 † · Public code repository; the first public date was not retrieved this round and the last commit is 2023-10-16 (checkpoint usage documentation) source
  • Weights release: No sourced date† · Only a 10% subset checkpoint is public; its upload date could not be retrieved from the host's interface source
Latest update · journal
Basis: Journal publication
Journal publication; the code repository's last commit is 2023-10-16 (checkpoint usage documentation). This is the latest sourced substantive event source
The most recent sourced event, not a claim that nothing changed since.
StatusMaintained only · No substantive event after 2023-10, though it is still widely used as a baseline
OpennessCode: Open · Apache-2.0
Weights: partial · Apache-2.0 (model card)
Code is open; the public weights are a subset model trained on 10% of each corpus, and the full large-model weights are not released
Benchmarks reported1 counted (self 1)
Wet-lab validationNone recorded
Unverified fields† release_milestones.code_release, release_milestones.weights_release, background.org_profile
As of2026-11-16

What it does

Treats molecule strings as text, using a twelve-layer Transformer with linear attention and rotary position embeddings so that billion-scale pretraining is feasible, then fine-tunes on downstream property tasks; the paper argues the representation captures molecular geometry and structure. Unlike graph models such as MolE it needs only strings rather than graphs or conformations, and unlike Uni-Mol2 it does not depend on 3D conformations.

Benchmarks reported

Every row is one pairing from the survey's relation file; a score belongs to the benchmark page's ranking, where it is shown with metric, eval set, source type and date. 1 of 1 pairings are counted.

BenchmarkRelationEval set / protocolSource typeNote and evidence
03 MoleculeNet (+OGB)Developer report, official protocol—Self-reportedMoleculeNet scaffold/random splits · evidence

Other evaluations (outside the 44)

  • Geometry probe of the representation (Own task set) Self-reported: The authors use self-built analyses to argue the string representation carries geometric and structural information
  • Corpus-scale controls (Own task set) Self-reported: Controls across pretraining corpus fractions; the public release corresponds to the 10% subset setting

Developers report: linear attention makes billion-scale string pretraining feasible, and after fine-tuning it beats the supervised graph-neural-network baselines of the time on public property tasks.

Independent evidence: No independent evidence found. No independent evaluation of the model was found, and task-group numbers in third-party papers are not board results

Training data

Group in Figure 7: Public data only.

ClassValueDetail
a. Public databasesYesUp to 1.1 billion molecule strings from two public compound libraries; the public version uses 10% of each source
b. Private or licensed dataNo source
c. Own wet-lab data used for trainingNo
d. Wet lab used only for validationNoThe paper reports no wet-lab validation source

Wet-lab validation

None. The developers report no wet-lab validation.

Background

  • Organization: The developer is the research division of a large US technology company, centred in New York State; its careers page claims over three thousand researchers across twelve labs on six continents (the page is undated). A change of research director appears only in second-hand sources with inconsistent months and is not concluded here.
  • Funding: Not disclosed
  • Products and partnerships: The company's chemistry foundation-model line includes this model, a later decoder-only generative model and a target-specific generative model, plus a public molecular foundation-model site; on-site comparisons are MolE (graph) and Uni-Mol2 (3D).
  • Related models: MolE, Uni-Mol2, MiniMol

Reading notes

  1. These are 2022 results that later foundation models use as a baseline, yet the full large-model weights are not public, so independent reproduction is hard.
  2. MoleculeNet datasets are small and split-sensitive, and studies report that most pretrained embeddings do not beat classical fingerprints.
  3. None of the 22 dataset pages of the official TDC ADMET board carries an entry for this model (checked 2026-11-16), so task-group numbers in third-party papers are not board results.
  4. The comparison baselines are weak (supervised graph networks only), so cross-paper comparisons need fingerprint baselines added.

Caveats

  1. The full weights are not public; only the 10% subset model is available.
  2. First dates for code and weights could not be retrieved this round and are marked unverified.
  3. All benchmark results are author-reported with no board entry.
  4. No substantive update after 2023-10.

Sources

  1. Journal paper 2022 (journal)
  2. Preprint (figures follow v3) (preprint)
  3. Code repository (repository)
  4. Public subset weights and model card (package registry)
  5. Another task-group leaderboard overview (used to rule out a pairing) (leaderboard)

Corrections

2 corrections
FieldWasNowBasisFound
first_release2022 (journal publication year)2021-06-17(arXiv v1)source2026-11-16
benchmarks_reportedCandidate pairing TDC ADMET task groupNot a pairing: none of the 22 dataset pages on the official board has an entry for this modelsource2026-11-16

Part of the AI for Bio benchmarks survey · all 112 models · open in the explorer. Record as of 2026-11-16.