{"id":"z3-adsorption-delta-campaign-round4","title":"Z3 Δ-Learned Adsorption Accuracy — Round-4 Results","subtitle":"Refuted — 128/128 baseline measurements; every validation-selected correction worsened the holdout (2.27–5.91 eV vs the 0.1 eV gate).","category":"validation","tags":["z3","adsorption","delta-learning","catbench","round-4","featured"],"source":"articles/docs/validation/z3-adsorption-delta-campaign-round4-results.md","lang":"en","words":758,"readMinutes":3,"toc":[{"depth":2,"text":"Preregistered question","id":"preregistered-question"},{"depth":2,"text":"Panel and protocol","id":"panel-and-protocol"},{"depth":2,"text":"Baseline result: systematic underbinding, panel-wide","id":"baseline-result-systematic-underbinding-panel-wide"},{"depth":2,"text":"Δ-correction: fitted, selected, scored — and refuted","id":"correction-fitted-selected-scored-and-refuted"},{"depth":2,"text":"What stands after refutation","id":"what-stands-after-refutation"},{"depth":2,"text":"Receipts","id":"receipts"}],"html":"

Z3 Δ-Learned Adsorption Accuracy — Round-4 Campaign Results

Status: completed campaign · verdict refuted (as preregistered)\nCampaign: discovery.round-4.z3-adsorption.v1 · executed 2026-07-19 (run z3-20260719)\nGate: corrected adsorption-energy MAE ≤ 0.1 eV against published DFT references on a 20-candidate holdout\nVerdict: all four available models FAIL — corrected holdout MAE 2.27–5.91 eV; every validation-selected correction made the holdout worse than the raw baseline

\n

Preregistered question

Can a Δ-learned hybrid stack — a foundation MLIP plus a small correction model fitted on six chemistry-family training systems and selected on six validation systems — reach the ≤ 0.1 eV adsorption-energy accuracy catalyst screening requires, on 20 held-out adsorbate–surface pairs it never saw during fitting, selection, or tuning?

\n

Panel and protocol

\n

Baseline result: systematic underbinding, panel-wide

Signed error = model − reference (positive = underbound). Holdout = the 20 confirmatory candidates.

\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelFull-panel MAEFull-panel biasHoldout MAE
chgnet1.67 eV+1.13 eV0.69 eV
mace-mp-medium3.43 eV+3.42 eV2.11 eV
mace-mp-small4.29 eV+4.28 eV3.24 eV
mace-mpa-0-medium5.31 eV+5.29 eV4.27 eV

Errors are not symmetric scatter: the MACE family underbinds essentially every system, with per-candidate errors from −1.1 eV to +25.6 eV, growing with adsorbate size and varying by family (plastics largest, polyurethanes smallest). The physical reading is dispersion blindness: the references stabilize large adsorbates partly through D2 dispersion, physics the foundation models' bulk-crystal training distribution does not contain. First datapoint anatomy (mace-mp-small × biomass_ni_mol1): gas molecule +1.39 eV, clean Ni(111) slab −12.45 eV, complex −1.24 eV — the slab/complex terms nearly cancel and the residual +9.8 eV sits entirely in the interface bond.

\n

Δ-correction: fitted, selected, scored — and refuted

A fixed correction menu was fitted per model on delta_train only and selected on delta_validation only (tools/build_z3_delta_correction.py; report data/candidates/z3/delta-correction-report.json + .sha256):

\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelSelected form (by validation MAE)Validation MAEBaseline holdout MAECorrected holdout MAEGate
chgnetC: family size-linear1.12 eV0.69 eV2.27 eV✗
mace-mp-mediumA: global constant3.26 eV2.11 eV5.01 eV✗
mace-mp-smallA: global constant3.98 eV3.24 eV5.00 eV✗
mace-mpa-0-mediumB: family constant4.03 eV4.27 eV5.91 eV✗

Every corrected holdout is worse than its own raw baseline. A global shift overshoots the small-error families into large negative errors; family/size-linear fits trained on two giant plastics systems extrapolate badly across the size distribution. The conclusion is structural, not anecdotal: the baseline error is not a uniform bias — it is a structured, family- and size-dependent field spanning −1 to +26 eV — and a six-point fit budget cannot estimate a generalizable correction over it. The Z3 Δ-learning hypothesis as preregistered is refuted on the holdout it froze.

\n

What stands after refutation

\n

Receipts

\n"}