{"id":"z1-barrier-campaign-round4","title":"Z1 Barrier-Accurate MLIP Campaign — Round-4 Results (Two Precision Chains)","subtitle":"Refuted twice — all four models miss the 40 meV gate (135–243 meV) identically under float32 and float64; systematic under-prediction.","category":"validation","tags":["z1","barriers","neb","batteries","round-4","featured"],"source":"articles/docs/validation/z1-barrier-campaign-round4-results.md","lang":"en","words":608,"readMinutes":3,"toc":[{"depth":2,"text":"Preregistered question","id":"preregistered-question"},{"depth":2,"text":"Panel and protocol","id":"panel-and-protocol"},{"depth":2,"text":"Results: both precision chains","id":"results-both-precision-chains"},{"depth":2,"text":"The shape of the failure","id":"the-shape-of-the-failure"},{"depth":2,"text":"Receipts","id":"receipts"}],"html":"

Z1 Barrier-Accurate MLIP Campaign — Round-4 Results (Two Precision Chains)

Status: completed campaign · verdict refuted — independently on two precision chains\nCampaign: discovery.round-4.z1-barriers.v1 · executed 2026-07-19 (float32 chain and float64 chain, same locked panel)\nGate: migration-barrier MAE ≤ 40 meV against published DFT-NEB references across 30 chemistry-held-out paths\nVerdict: all four available models FAIL, 3.4–6× over threshold, with a systematic under-prediction bias — unchanged under float64

\n

Preregistered question

Can the declared available foundation MLIPs (chgnet 0.4.2; mace-torch 0.3.16 small / medium / mpa-0-medium) predict solid-state ion migration barriers accurately enough for battery-electrolyte screening — a mean absolute error of 40 meV or better against DFT-NEB references on 30 held-out chemical systems?

\n

Panel and protocol

\n

Results: both precision chains

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Modelfloat32 MAEfloat64 MAEPaths (both chains)Gate
mace-mpa-0-medium135.0 meV135.0 meV28/30✗
mace-mp-small152.0 meV151.9 meV26/30✗
mace-mp-medium174.7 meV174.7 meV29/30✗
chgnet242.5 meV242.5 meV28/30✗

The first chain ran at float32 — against MACE vendor guidance for geometry optimization — a protocol defect found in post-execution review, fixed once (barrier row only; PR #32), and re-measured end-to-end at float64. The verdict is precision-independent: per-model MAEs are identical to within 0.1 meV. The miss is model error, not numerical artifact. Both chains stand as executed evidence with separate artifact prefixes (z1/campaign/ and z1/campaign-float64/); checkpoint contexts bind calculator dtype (PR #36), so no float32 prediction can leak into a float64 record.

\n

The shape of the failure

\n

Receipts

\n"}