{"id":"savings-delta-multifidelity","title":"Δ-Learning, Multi-Fidelity, and Transfer-Learning Corrections","subtitle":"Correction layers that learn the gap between cheap and expensive electronic-structure methods.","category":"references","tags":["literature-review","savings-stack","delta-learning","multi-fidelity","transfer-learning"],"source":"articles/lit-review/savings-delta-multifidelity.md","lang":"en","words":2420,"readMinutes":11,"toc":[{"depth":2,"text":"1. Executive result","id":"1-executive-result"},{"depth":2,"text":"2. Savings table","id":"2-savings-table"},{"depth":2,"text":"3. Proven vs claimed","id":"3-proven-vs-claimed"},{"depth":2,"text":"4. Openings for Lupine","id":"4-openings-for-lupine"}],"html":"
\n

Provenance: explore agent agent-15 (director-commissioned deep research, swarm of 7, 2026-07-21) — materialized verbatim. Quantitative claims are as reported by the research agent from sources it accessed; see citations inline. Citation-verification pass pending before any external publication.

\n
\n

Chapter Section: Δ-Learning, Multi-Fidelity, and Transfer-Learning Corrections — Insider's Digest

Evidence cut: 2026-07-21. All citations below were accessed this session; each is annotated with what was actually read (abstract / full text / snippet).

\n

1. Executive result

\n

2. Savings table

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
TechniqueWhat cost it removesMeasured savings (system/size context)Accuracy cost / failure modeCitation (accessed as)Evidence
Δ-ML kernel correction (semiempirical→DFT)High-level QC labels for trainingChemical accuracy on enthalpies of 134k organic molecules training on 1–10% of them (GDB-9-derived set)Residual must be smooth/small; kernel scaling O(N³); energies only in originalRamakrishnan 2015, arXiv:1503.04987; JCTC 11:2087, DOI 10.1021/acs.jctc.5b00099 (abstract)strong (foundational, widely replicated)
Multi-fidelity CQML (HF/MP2/CCSD(T) stack)~90–99% of gold-standard labels~100 CCSD(T)/cc-pVDZ labels instead of thousands for ~1 kcal/mol atomization-energy accuracy on ~7k moleculesNeeds nested fidelity structure; KRR cost grows with multi-level dataZaspel 2019, arXiv:1808.02799v2; JCTC 15:1546, DOI 10.1021/acs.jctc.8b00832 (abstract)strong
Transfer learning of NNP (DFT→CCSD(T)*/CBS)High-fidelity data + training timeANI-1ccx: 500k CC labels on top of 5.2M DFT; 23% lower RMSD than training on the CC data alone; retrain 30 min vs 4 h per net; torsion MAD halved vs DFT-trained model (0.23 vs 0.47 kcal/mol median)Still needs ~10⁵ high-level points for broad chemistry; paper itself notes Δ-learning reaches similar accuracy but doubles inference costSmith 2019, Nat. Commun. 10:2903, DOI 10.1038/s41467-019-10827-4 (full text)strong
Δ + transfer-learning hybrid on semiempirical baselineDFT/CC cost at inferenceAIQM1: near-G4/G4MP2 heats of formation (MAD 0.9 kcal/mol, CHNO set; parent SQM 2.6) at semiempirical speed; ~1000×+ vs DFT geometry optimization (14 s vs 31 min×32 cores for C₆₀)Neutral closed-shell H/C/N/O only; fails where training data sparse (H₂ error −2.9 kcal/mol)Zheng 2021, Nat. Commun. 12:7022, DOI 10.1038/s41467-021-27340-2 (full text)strong
Δ-ML PES correction (DFT→CCSD(T), PIP fits)CC gradients/labels for full PESDFT-based PES elevated to near-CCSD(T) for molecules up to 15 atoms (H₃O⁺…acetylacetone, tropolone; ethanol study across PBE/M06/M06-2X/PBE0+MBD); improvement even in gradients without CC gradients in fitSingle-molecule PESs; needs dense low-level sampling firstNandi 2021, JCP 154:051102, DOI 10.1063/5.0038301 (citation+snippet); Nandi 2024, arXiv:2407.20050; JCTC 20:8807, DOI 10.1021/acs.jctc.4c00977 (abstract)strong (Bowman-group series, replicated)
Multi-fidelity graph networks (PBE→expt)Scarce experimental/high-fidelity labels22–45% lower MAE on experimental band gaps of ordered+disordered materials using PBE gaps as low fidelityProperty-prediction, not PES; gains shrink as high-fidelity data growsChen et al. 2021, arXiv:2005.04338v3; Nature Computational Science 1:46–53, DOI 10.1038/s43588-020-00002-x (abstract)strong
Implicit delta multi-task NNP (semiempirical→DFT)High-fidelity labelsIDLe matches single-fidelity baseline with up to 50× less high-fidelity data (SPICE-like molecular datasets; 11M semiempirical points released)Single preprint; multi-head architecture required; Δ implicit, no physics handle at inferenceTossou et al. 2024, arXiv:2412.06064v1 (abstract)claim (single preprint)
uMLIP fine-tuning to higher level of theoryFrom-scratch training data & timeMatterSim water: 30 revPBE0-D3 configs ≈ from-scratch-900 (97% fewer labels; reproduces RDF/ADF + diffusion within 20% of expt); Li₂B₁₂H₁₂ active learning: 15% of dataNumbers from model authors; best-case system; fine-tune inherits pretrained biasMatterSim, arXiv:2405.04967v2 (full text)strong (but single-team)
Multi-task pretrain → fine-tune (DPA-2)Downstream DFT labels1–2 orders of magnitude less downstream data across tasks (alloys, SSE, ferroelectrics, drug molecules); 2 orders on H₂O-PBE0TS; distillation recovers ~2 orders inference speedGains shrink as downstream data grows; needs overlap between pretraining and target chemistryDPA-2, arXiv:2312.15492v2 (full text)strong
Cross-dataset fine-tuning of big pretrained models (OMat24→MPtrj)Re-training on 110M structures per userOMat24-pretrained eSEN/eqV2 fine-tuned to MPtrj/sAlex: Matbench Discovery F1 0.925, energy-above-hull MAE 18 meV/atom (best at time of writing)Fine-tuning on relaxation data reintroduces systematic softening (own analysis)OMat24, arXiv:2410.12771v2 (full text)strong
TS-focused uMLIP fine-tuning for barriers (CHGNet)DFT NEB/AIMD cost at scaleMigration-barrier MAE 0.23–0.24 → 0.07–0.09 eV on Li-ion conductors; HT-NEB screening of quaternary Li compounds; claimed 100–1000× vs DFT NEB; <1k DFT structures per modelFine-tune set must contain transition states (chicken-and-egg: pretrained NEB generates them); 0.07–0.09 eV residual error still large for kineticsLian 2025, arXiv:2507.02334; JMCA 13:34918, DOI 10.1039/D5TA05355B (abstract + RSC text snippet)moderate-strong
Small-set uMLIP fine-tuning for phonons/EXAFS (CHGNet)Compound-specific DFT datasets~100 DFT structures fine-tune CHGNet to DFT-level phonons/thermal disorder for 2H-WS₂/MoS₂; even 1 structure partially viableCompound-specific; force MAE floor ~100 meV/ÅŽguns 2025, arXiv:2509.08498v1 (abstract)moderate-strong
Fine-tune vs from-scratch, controlled comparisons (MACE)Training data + convergence timeFine-tuned MACE beats from-scratch at every data fraction (10–100%) on LGPS (final force RMSE ~15 meV/Å); Mo elastic constants: C11 error 45.9%→2.6%, C44 56.9%→14.8%, ν 48.3%→3.5%; beats uncertainty-filtered from-scratch on Si OOD setSingle-GPU tutorial-scale; hyperparameter-sensitive; forgetting needs mitigations (num_samples_pt, multi-head replay)Liu/Zeng et al. 2025, arXiv:2506.21935v2 (full text)moderate (preprint tutorial, very thorough)
Architecture-agnostic fine-tune sweep (MACE/GRACE/SevenNet/MatterSim/ORB)System-specific ab initio accuracyFine-tuning improves force accuracy 5–15×, energy accuracy 2–4 orders of magnitude, across 7 compounds, using equidistantly sampled short AIMD framesSingle preprint; "near-ab initio" claim system-dependentHänseroth 2025, arXiv:2511.05337v1 (abstract)claim→moderate
GPR Δ-model on uMLIP's own embeddingsRe-fitting / full fine-tuningCHGNet errors on sulfur-overlayer global optimization (Cu/Ag/Au(111)) corrected by sparse-GPR Δ-model trained on small DFT sets; same bias found in MACE-MP-0, SevenNet-0, ORB-v2Correction is global/static, not query-specific; needs active-learning loop for GOHammer 2025, arXiv:2502.21179v2 (abstract); Pitfield 2026, arXiv:2507.18485; PCCP 28:912, DOI 10.1039/D5CP04302F (abstract snippet)moderate
Periodic (iterative) fine-tuningDFT labels while avoiding OOD failurePeriodic fine-tuning generalizes where "naive" single-trajectory fine-tuning fails OOD (uMLIP MD bias study)Fine-tuning can inject model bias; needs uncertainty monitoring (Q-residual)Wong 2026, arXiv:2603.10159v2; JCTC, DOI 10.1021/acs.jctc.6c00425 (abstract)moderate
Fine-tune across DFT settings (UMA)Re-generating data at consistent settingsUMA models fine-tuned on MPtrj/sAlex to align DFT settings for materials evals; single un-fine-tuned model ≈ specialized models (abstract claim)No public data-efficiency numbers for fine-tuning in accessed textUMA, arXiv:2506.23971v1/v2 (abstract + PDF snippet)moderate (fine-tune numbers) / strong (zero-shot claim)
Data-cost benchmarking of Δ vs MFMLMisallocated QC budgetMFML beats Δ-ML when many predictions needed; Δ-ML (and MFΔML) wins for few predictions (QeMFi benchmark, 3 properties)Kernel methods; small moleculesVinod & Zaspel 2024/25, arXiv:2410.11391v3 (abstract)strong (guidance, not a technique)

3. Proven vs claimed

Replicated and trusted (multi-group, multi-year):

\n\n

Single-paper claims (plausible, not yet independently replicated):

\n\n

Marketing / read with care:

\n\n

4. Openings for Lupine

    \n
  1. Beat the fine-tuned-NEB baseline on its own metric, with 10–50× fewer labels. The strongest published barrier-correction result (Lian et al., JMCA 2025) needs on the order of 10³ transition-state DFT labels per model family and still carries 0.07–0.09 eV barrier MAE. Lupine's sparse-anchor protocol (4–6 DFT images near the model-predicted saddle, per path) plus theorem-gated runtime correction targets per-path errors rather than amortized global ones. A head-to-head on their NASICON/LATP panel — labels-per-barrier vs barrier MAE — is the cleanest possible bake-off: fine-tuning pays the label cost upfront and amortizes over many paths of one chemistry; Lupine pays ~5 labels per path and needs no retraining when the chemistry changes. Union anchors (79% fewer DFT calls on the 29-path panel) stack multiplicatively on top.
    2. \n
  2. Turn Hammer-style Δ-GPR from a static global correction into a theorem-gated local one. The Aarhus work (arXiv:2502.21179, arXiv:2507.18485) proves that (i) uMLIP errors are systematic and representation-learnable, and (ii) a sparse-GPR Δ-model on the model's own embeddings can correct global optimization with a small active DFT set. But their correction is a single global surface with no per-query guarantee and was demonstrated for structure optimization, not barriers. Lupine can consume the same signal (embedding-space error predictability) while gating each corrected quantity — saddle height, curvature — against Lean-formalized physical theorems, and falling back to DFT anchors when the gate fails. That is a strict superset of their scheme, aimed at the barrier/NEB regime they leave open.
    3. \n
  3. Use multi-fidelity optimal-budget theory to make "sparse" anchors provably sparse. Zaspel's CQML and the Vinod–Zaspel data-cost benchmarking (arXiv:2410.11391) formalize exactly our allocation problem: how many evaluations at each fidelity minimize total QC cost for a target accuracy. Lupine's current heuristic (~4–6 anchors per band) is a hand-tuned point in that space; a MFML-style cost model treating {uMLIP ensemble, fine-tuned uMLIP, DFT anchor} as three fidelities could derive the anchor count per path from measured inter-fidelity correlation — and the benchmarking paper's key finding (Δ-ML-style two-level schemes win precisely when few high-fidelity evaluations are available) is theoretical cover for the sparse-anchor regime where Lupine operates.
    4. \n
  4. Position against fine-tuning's documented failure modes as the "don't retrain, correct at runtime" alternative. Three verified failure modes are exploitable openings: fine-tuning reintroduces PES softening when the fine-tune set is relaxation-heavy (OMat24 v2 analysis); naive fine-tuning fails out-of-distribution and can inject bias (Wong et al. 2026); pretrained representations carry systematic chemistry biases (Hammer 2025). Each fine-tune also costs a GPU training run per model per chemistry, and with N uMLIPs in a union-anchor pool that cost multiplies — whereas Lupine's anchors are shared across models by construction. A "theorem-gated runtime correction vs per-model fine-tuning" comparison, including the catastrophic-forgetting and softening-reintroduction cases, would be a differentiating chapter figure: fine-tuning is the right tool for stable, high-volume single chemistry; Lupine wins for breadth, barriers, and guaranteed correctness.
    5. \n
\n

Honest gaps in this subdomain: no public study yet quantifies fine-tuning data-efficiency for UMA-scale models (Meta's fine-tuning numbers are not in the accessible text — marked accordingly above); Bogojeski et al. 2020 (DTNN Δ-learning) could not be fetched (Nature redirect loop) and was dropped rather than cited; Pilania/Batra multi-fidelity co-kriging classics (hafnia, ACS AMI 2019, DOI 10.1021/acsami.9b02174) were only seen as citations, not read, and are excluded from quantitative claims. The subdomain is not thin — it is currently the hottest part of the literature — but truly controlled fine-tune-vs-scratch experiments remain concentrated in preprints from model authors' own teams.

\n"}