{"id":"supercomputing-atomistic-history","title":"Atomic-Scale Simulation on Supercomputers","subtitle":"Machine history and capex per PFLOP, and what four decades of hardware bought atomistic simulation.","category":"references","tags":["literature-review","savings-stack","supercomputing","history","hpc"],"source":"articles/lit-review/supercomputing-atomistic-history.md","lang":"en","words":1595,"readMinutes":7,"toc":[{"depth":2,"text":"1. Machine table","id":"1-machine-table"},{"depth":2,"text":"2. Workload evolution (with citations)","id":"2-workload-evolution-with-citations"},{"depth":2,"text":"3. Timeline (compressed)","id":"3-timeline-compressed"},{"depth":2,"text":"4. Cost & access notes","id":"4-cost-access-notes"},{"depth":2,"text":"Items flagged as uncertain or unverifiable","id":"items-flagged-as-uncertain-or-unverifiable"}],"html":"<blockquote>\n<p><strong>Provenance:</strong> explore agent <code>agent-11</code> (director-commissioned deep research, backdrop research, 2026-07-21) — materialized verbatim. Quantitative claims are as reported by the research agent from sources it accessed; see citations inline. Citation-verification pass pending before any external publication.</p>\n</blockquote>\n<h1 id=\"atomic-scale-simulation-on-supercomputers-expert-digest\">Atomic-Scale Simulation on Supercomputers: Expert Digest</h1><p>All figures below are sourced; derived numbers are labeled <strong>[derived]</strong>, and anything not cleanly verifiable is flagged <strong>[uncertain]</strong>. Machine performance numbers are TOP500 HPL (Rmax) unless noted.</p>\n<hr>\n<h2 id=\"1-machine-table\">1. Machine table</h2><div class=\"table-wrap\"><table><thead><tr>\n<th>Machine</th>\n<th>Site / debut</th>\n<th>Peak / HPL (FP64)</th>\n<th>Architecture</th>\n<th>Cost</th>\n<th>Power</th>\n<th>Notes for atomistic sim</th>\n</tr>\n</thead><tbody><tr>\n<td data-label=\"Machine\"><strong>Tianhe-2</strong></td>\n<td data-label=\"Site / debut\">NSCC Guangzhou; #1 June 2013 – Nov 2015</td>\n<td data-label=\"Peak / HPL (FP64)\">54.9 PF peak / 33.86 PF HPL</td>\n<td data-label=\"Architecture\">16,000 nodes × (2× Intel Xeon E5-2692 + 3× Xeon Phi 31S1P), 3.12M cores</td>\n<td data-label=\"Cost\">¥2.4B ≈ <strong>$390M</strong> (<a href=\"https://en.wikipedia.org/wiki/Tianhe-2\">Wikipedia/Dongarra report</a>)</td>\n<td data-label=\"Power\">17.6 MW; 24 MW with cooling</td>\n<td data-label=\"Notes for atomistic sim\">2015 US export ban blocked its upgrade → rebuilt as Tianhe-2A with domestic Matrix-2000 (61.4 PF HPL)</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Sunway TaihuLight</strong></td>\n<td data-label=\"Site / debut\">NSCC Wuxi; #1 June 2016 – June 2018</td>\n<td data-label=\"Peak / HPL (FP64)\">125.4 PF peak / 93.01 PF HPL</td>\n<td data-label=\"Architecture\">40,960× SW26010 (260 cores @1.45 GHz), 10.65M cores, <strong>no accelerators</strong> (<a href=\"https://www.netlib.org/utk/people/JackDongarra/PAPERS/sunway-report-2016.pdf\">Dongarra report</a>, <a href=\"https://www.glennklockwood.com/garden/systems/Sunway-TaihuLight\">Klockwood notes</a>)</td>\n<td data-label=\"Cost\">¥1.8B ≈ <strong>$270–273M</strong> incl. R&amp;D (<a href=\"https://www.datacenterdynamics.com/en/analysis/chinas-sunway-taihulight-named-worlds-most-powerful-computer-uses-homegrown-tech/\">DCD</a>, <a href=\"http://www.scio.gov.cn/news_0/202209/t20220921_415792.html\">SCIO</a>)</td>\n<td data-label=\"Power\">15.37 MW (HPL)</td>\n<td data-label=\"Notes for atomistic sim\">First all-domestic Chinese #1; later a proving ground for ML-potential MD (see §2)</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Summit</strong></td>\n<td data-label=\"Site / debut\">ORNL OLCF; #1 June 2018 – June 2020</td>\n<td data-label=\"Peak / HPL (FP64)\">200.8 PF peak / 148.6 PF HPL</td>\n<td data-label=\"Architecture\">4,608 nodes × (2× IBM POWER9 22c + 6× NVIDIA V100)</td>\n<td data-label=\"Cost\"><strong>~$200M</strong> (<a href=\"http://oakridgetoday.com/2019/06/17/ornls-summit-remains-worlds-most-powerful-supercomputer/\">Oak Ridge Today</a>)</td>\n<td data-label=\"Power\">~13 MW full system (<a href=\"https://www.nextplatform.com/2017/09/19/power9-rollout-begins-summit-sierra/\">Next Platform</a>); HPL power ~10 MW <strong>[uncertain]</strong></td>\n<td data-label=\"Notes for atomistic sim\">Machine that launched MLIP-at-scale: 2020 Gordon Bell DPMD run; 3.3 mixed-precision &quot;exaops&quot; advertised</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Sierra</strong></td>\n<td data-label=\"Site / debut\">LLNL (NNSA); online Oct 2018, #2 Nov 2018</td>\n<td data-label=\"Peak / HPL (FP64)\">125.7 PF peak / 94.6 PF HPL</td>\n<td data-label=\"Architecture\">4,320 nodes × (2× POWER9 + 4× V100), 17,280 GPUs (<a href=\"https://hpc.llnl.gov/hardware/compute-platforms/sierra\">LLNL</a>, <a href=\"https://dl.acm.org/doi/10.1145/3721145.3734532\">ACM ICS&#39;25</a>)</td>\n<td data-label=\"Cost\">Not separately disclosed <strong>[uncertain]</strong> (CORAL program total ~$525M, <a href=\"https://fedscoop.com/energy-dept-aurora-supercomputer-intel-cray/\">FedScoop</a>)</td>\n<td data-label=\"Power\">~11–13 MW <strong>[uncertain]</strong></td>\n<td data-label=\"Notes for atomistic sim\">Classified stockpile stewardship; decommissioned (<a href=\"https://hpc.llnl.gov/hardware/compute-platforms/sierra\">LLNL page</a>)</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Fugaku</strong></td>\n<td data-label=\"Site / debut\">RIKEN R-CCS Kobe; #1 June 2020 – June 2022</td>\n<td data-label=\"Peak / HPL (FP64)\">537 PF peak / 442 PF HPL</td>\n<td data-label=\"Architecture\">158,976× Fujitsu A64FX (48-core Arm SVE + HBM2), 7.63M cores, <strong>no GPUs</strong> (<a href=\"https://www.datacenterdynamics.com/en/news/fugaku-remains-worlds-fastest-supercomputer-in-latest-top500/\">DCD</a>)</td>\n<td data-label=\"Cost\">Reported <strong>~$1B</strong> total program (<a href=\"https://cacm.acm.org/news/fugaku-takes-the-lead/\">CACM</a>) <strong>[reported, not official]</strong></td>\n<td data-label=\"Power\">~30 MW (Fujitsu spec: 30–40 MW, <a href=\"https://global.fujitsu/en-global/technology/research/fugaku\">Fujitsu</a>)</td>\n<td data-label=\"Notes for atomistic sim\">First machine to break 2 EF on HPL-AI (mixed precision); shared use from 9 Mar 2021</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>LUMI</strong></td>\n<td data-label=\"Site / debut\">CSC Kajaani, Finland; full system Nov 2022</td>\n<td data-label=\"Peak / HPL (FP64)\">~530–552 PF peak <strong>[uncertain, sources differ]</strong> / 379.7 PF HPL</td>\n<td data-label=\"Architecture\">LUMI-G: 2,560 nodes × (1× AMD &quot;Trento&quot; 64c + 4× MI250X); HPE Cray EX (<a href=\"https://sciencebusiness.net/network-updates/csc-supercomputer-ranked-among-worlds-fastest\">CSC/ScienceBusiness</a>, <a href=\"https://www.rankred.com/fastest-supercomputers-in-the-world/\">RankRed</a>)</td>\n<td data-label=\"Cost\"><strong>€200M</strong> lifecycle (<a href=\"https://www.eurohpc-ju.europa.eu/lumi-new-eurohpc-world-class-supercomputer-finland-2020-10-21_en\">EuroHPC</a>; total ecosystem budget &gt;€600M per <a href=\"https://indico.cern.ch/event/1466360/contributions/6325288/attachments/3007776/5302233/CSC-update-300125.pdf\">CSC</a>)</td>\n<td data-label=\"Power\">7.1 MW, hydro-powered with waste-heat reuse (<a href=\"https://www.datacenterdynamics.com/en/analysis/lumi-europes-fastest-ai-supercomputer-finland/\">DCD</a>)</td>\n<td data-label=\"Notes for atomistic sim\">Europe&#39;s flagship GPU machine; major MLIP/materials user community</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Frontier</strong></td>\n<td data-label=\"Site / debut\">ORNL OLCF; <strong>first exascale</strong>, #1 June 2022 – Nov 2024</td>\n<td data-label=\"Peak / HPL (FP64)\">1.685 EF peak / 1.102 EF HPL (June 2022); 1.353 EF (Nov 2024)</td>\n<td data-label=\"Architecture\">9,408 nodes × (1× AMD EPYC &quot;Trento&quot; 64c + 4× MI250X), HPE Cray EX235a (<a href=\"https://www.arxiv.org/pdf/2506.19019\">arXiv survey</a>)</td>\n<td data-label=\"Cost\"><strong>$600M</strong> (<a href=\"https://tipsmake.com/10-fastest-supercomputers-iocqo\">widely reported</a>)</td>\n<td data-label=\"Power\">21.1 MW during HPL run (<a href=\"https://www.osti.gov/servlets/purl/2429796\">OSTI/IEEE paper</a>); 52.2 GF/W</td>\n<td data-label=\"Notes for atomistic sim\">Host of 2023 and 2024 Gordon Bell atomistic prizes (§2)</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>Aurora</strong></td>\n<td data-label=\"Site / debut\">ANL ALCF; exascale verified May 2024</td>\n<td data-label=\"Peak / HPL (FP64)\">1.98 EF peak / 1.012 EF HPL (on 87% of system)</td>\n<td data-label=\"Architecture\">HPE Cray EX; Intel Xeon Max 9470 + Intel Data Center GPU Max (&quot;Ponte Vecchio&quot;), 6 GPUs/node (<a href=\"https://www.hpe.com/us/en/newsroom/press-release/2024/05/hewlett-packard-enterprise-delivers-second-exascale-supercomputer-aurora-to-us-department-of-energys-argonne-national-laboratory.html\">HPE</a>)</td>\n<td data-label=\"Cost\"><strong>$500M+</strong> contract (<a href=\"https://www.energy.gov/articles/us-department-energy-and-intel-build-first-exascale-supercomputer\">DOE, 2019</a>)</td>\n<td data-label=\"Power\">38.7 MW (<a href=\"https://en.wikipedia.org/wiki/Aurora_(supercomputer)\">Wikipedia</a>)</td>\n<td data-label=\"Notes for atomistic sim\">10.6 EF on HPL-MxP (mixed precision) — largest AI-capable system at delivery</td>\n</tr>\n<tr>\n<td data-label=\"Machine\"><strong>El Capitan</strong></td>\n<td data-label=\"Site / debut\">LLNL (NNSA); <strong>#1 since Nov 2024</strong>; dedicated 9 Jan 2025</td>\n<td data-label=\"Peak / HPL (FP64)\">2.746 EF peak / 1.742 EF HPL</td>\n<td data-label=\"Architecture\">11,136 nodes × 4× AMD MI300A APU (44,544 APUs; 24 Zen4 cores + 228 CDNA3 CUs each), 5.4 PB HBM3 (<a href=\"https://www.llnl.gov/article/51231/llnls-el-capitan-debuted-new-top500-list-worlds-most-powerful-supercomputers\">LLNL</a>, <a href=\"https://chinaaet.com/article/3000168703\">chinaaet/Top500 report</a>)</td>\n<td data-label=\"Cost\"><strong>$600M</strong> (<a href=\"https://caliber.az/en/post/el-capitan-us-breaks-supercomputing-barriers-with-new-600-million-machine\">Caliber.Az/AP</a>)</td>\n<td data-label=\"Power\">~30 MW; 58.89 GF/W Green500 (<a href=\"https://www.itpro.com/technology/what-is-exascale-computing-exploring-the-next-step-in-supercomputers\">IT Pro</a>)</td>\n<td data-label=\"Notes for atomistic sim\">Classified NNSA stockpile work; unclassified sibling Tuolumne exists</td>\n</tr>\n</tbody></table></div><hr>\n<h2 id=\"2-workload-evolution-with-citations\">2. Workload evolution (with citations)</h2><p><strong>Phase A — plane-wave/pseudopotential DFT era (–2015).</strong> Community codes VASP and Quantum ESPRESSO (Giannozzi et al., <em>JPCM</em> 21, 395502, 2009; <a href=\"https://arxiv.org/html/2510.01875\">cited in QE exascale papers</a>) dominated leadership allocations; routine DFT topped out at ~10³ atoms. Linear-scaling efforts (PEXSI, ~10–20k atoms; <a href=\"https://www.bnl.gov/nysds20/files/talks/lin_lin.pdf\">Lin Lin slides</a>) were the scale frontier. This era maps to Tianhe-2&#39;s Xeon+Phi architecture.</p>\n<p><strong>Phase B — GPU adoption + classical MD at scale (2016–2020).</strong> QE&#39;s exascale rewrite is documented in &quot;Quantum ESPRESSO toward the exascale&quot; (Giannozzi et al., <em>JCP</em> 152, 154105, 2020) and the follow-up JCTC 19, 6992 (2023). LAMMPS + Kokkos became the portability vehicle; DOE&#39;s Exascale Computing Project funded the <strong>EXAALT</strong> (Exascale Atomistics for Accuracy, Length, and Time) project, whose SNAP kernel re-engineering gave a 5× speedup on Summit (<a href=\"https://www.energy.gov/science/articles/fy-2021-advanced-scientific-computing-research-budget-request\">DOE FY2021 budget justification</a>).</p>\n<p><strong>Phase C — MLIP era (2018–2022).</strong></p>\n<ul>\n<li>Deep Potential MD method: Zhang et al., <em>PRL</em> 120, 143001 (2018).</li>\n<li><strong>2020 ACM Gordon Bell Prize</strong>: Jia, Wang, Chen, Lu, Lin, Car, E, Zhang, &quot;Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning,&quot; SC&#39;20 — 100M atoms at 91 PF (FP64) / 275 PF (mixed half precision), run on <strong>Summit</strong> (<a href=\"https://mags.acm.org/communications/february_2021/MobilePagedArticle.action?articleId=1654975\">ACM</a>; journal version: Lu et al., &quot;86 PFLOPS DPMD…,&quot; <em>Comput. Phys. Commun.</em> 259, 107624, 2021). Note: this prize run was on Summit, not Sunway — the Sunway association comes from the follow-up below.</li>\n<li><strong>10 billion atoms</strong>: Guo et al., &quot;Extending the limit of molecular dynamics with ab initio accuracy to 10 billion atoms,&quot; PPoPP 2022, on the new-generation Sunway system (<a href=\"https://arxiv.org/abs/2201.01446\">arXiv:2201.01446</a>) — the Sunway hardware generation is described in the paper; treat &quot;OceanLight-class prototype&quot; phrasing as <strong>[lightly uncertain]</strong>.</li>\n<li>Billion-atom classical-precision-scale MD: Nguyen-Cong et al., SC21 — 20B-atom SNAP/LAMMPS carbon run on all 27,900 Summit GPUs, 50 PF peak, 97% strong-scaling efficiency (<a href=\"https://www.consensus.app/papers/billion-atom-molecular-dynamics-simulations-of-carbon-at-nguyen-cong-belonoshko/0c366bdff8f656088bbc0269f4be1c57\">Consensus/SC21</a>).</li>\n<li>Universal MLIPs begin: <strong>M3GNet</strong> (Chen &amp; Ong, <em>Nat. Comput. Sci.</em> 2, 718–728, 2022), CHGNet (Deng et al., 2023).</li>\n</ul>\n<p><strong>Phase D — exascale + foundation models (2022–now).</strong></p>\n<ul>\n<li>Frontier exascale HPL: June 2022 (<a href=\"https://www.nextplatform.com/hpc/2022/05/30/at-long-last-hpc-officially-breaks-the-exascale-barrier/1646968\">Next Platform</a>).</li>\n<li><strong>2023 Gordon Bell Prize</strong>: DFT-FE team (incl. Motamarri, Gavini) — real-space DFT at 659.7 PF on Frontier, quasicrystals and alloy defects (<a href=\"https://www.acm.org/media-center/2023/november/gordon-bell-prize-2023\">ACM, 16 Nov 2023</a>, <a href=\"https://cds.iisc.in/dr-phani-motamarri-and-his-research-group-part-of-the-team-that-won-the-2023-acm-gordon-bell-prize/\">IISc</a>).</li>\n<li><strong>2024 Gordon Bell Prize</strong>: Barca et al. — &quot;Breaking the Million-Electron and 1 EFLOP/s Barriers: Biomolecular-Scale Ab Initio MD Using MP2 Potentials,&quot; &gt;1M electrons, &gt;1 EF/s sustained on Frontier (<a href=\"https://www.acm.org/media-center/2024/november/gordon-bell-prize-2024\">ACM, 21 Nov 2024</a>, <a href=\"https://www.unimelb.edu.au/newsroom/news/2024/november/melbourne-researchers-win-the-nobel-of-high-performance-computing\">Univ. Melbourne</a>).</li>\n<li>Foundation-model MLIPs: <strong>MACE-MP-0</strong> (Batatia et al., <a href=\"https://arxiv.org/pdf/2401.00096.pdf\">arXiv:2401.00096</a>, Dec 2023; published <em>JCP</em> 163, 184110, 2025, already &gt;1000 citations per <a href=\"https://pubs.aip.org/aip/jcp/article/163/18/184110/3372267/A-foundation-model-for-atomistic-materials\">AIP</a>).</li>\n<li>Frontier ParSplice+SNAP: 13,856 concurrent LAMMPS instances on ~75% of Frontier, 3.57×10⁹ atom-steps/s sustained for 1 h (<a href=\"https://dl.acm.org/doi/pdf/10.1145/3581784.3607089\">ACM SC paper</a>).</li>\n<li>Latest scale record: 1.62-trillion-atom MD with Neuroevolution Potential (NEP), &quot;ab initio accuracy,&quot; on China&#39;s new-generation intelligent supercomputer — 2026 <strong>preprint</strong> (<a href=\"https://arxiv.org/pdf/2604.24816\">arXiv:2604.24816</a>) <strong>[preprint, unrefereed]</strong>.</li>\n<li>Cross-machine portability proven: LAMMPS-Kokkos strong scaling on Frontier, Aurora, El Capitan, and Alps (<a href=\"https://arxiv.org/abs/2508.13523\">arXiv:2508.13523</a>).</li>\n</ul>\n<p><strong>AI-for-science programs.</strong> DOE&#39;s &quot;AI for Science, Energy and Security&quot; report (May 2023, <a href=\"https://science.osti.gov/-/media/ascr/pdf/Office-Hours/2024/ASCR-Overview---2024-03.pdf\">ANL/ASCR overview</a>); the <strong>FASST</strong> (Frontiers in AI for Science, Security and Technology) initiative (<a href=\"https://www.hpcuserforum.com/wp-content/uploads/2024/09/Rick-Stevens_ANL_FASST-Initiative_HPC-AI-User-Forum-Sept-2024.pdf\">ANL/HPC User Forum deck</a>). Atomistic-specific allocations: 2025 INCITE project &quot;exascale ab initio reactive MD (AIRMD)&quot; awarded 380,000 Frontier node-hours (<a href=\"https://doeleadershipcomputing.org/wp-content/uploads/sites/123/2024/11/2025INCITEFactSheets.pdf\">2025 INCITE fact sheets</a>); 2026 INCITE includes an &quot;Ab Initio Foundation Model&quot; MLIP project (<a href=\"https://doeleadershipcomputing.org/wp-content/uploads/sites/123/2026/05/2026INCITEFactSheets.pdf\">2026 fact sheets</a>).</p>\n<hr>\n<h2 id=\"3-timeline-compressed\">3. Timeline (compressed)</h2><ul>\n<li>2013–2015: Tianhe-2 #1 — end of the CPU/coprocessor DFT era; US export ban (2015) pushes China to domestic silicon.</li>\n<li>June 2016: Sunway TaihuLight #1, 93 PF, all-domestic SW26010.</li>\n<li>2018: Summit (#1) and Sierra (#2) — V100 GPU era; Deep Potential PRL published.</li>\n<li>2020: Fugaku #1 (442 PF, Arm, no GPU); SC20 Gordon Bell: 100M-atom DPMD on Summit.</li>\n<li>2022: Frontier breaks exascale (1.102 EF); PPoPP 10B-atom DPMD on new Sunway; M3GNet.</li>\n<li>2023: DFT-FE Gordon Bell on Frontier (659.7 PF); MACE-MP-0 preprint; DOE AI4S report.</li>\n<li>2024: Aurora verified exascale (May); El Capitan #1 at 1.742 EF (Nov); MP2 1M-electron Gordon Bell on Frontier.</li>\n<li>2025–2026: El Capitan dedicated; MACE-MP-0 published; 1.62T-atom MD preprint; FASST / AI4S INCITE projects.</li>\n</ul>\n<hr>\n<h2 id=\"4-cost-amp-access-notes\">4. Cost &amp; access notes</h2><p><strong>Capex per peak PFLOP [derived — capex ÷ Rpeak]:</strong></p>\n<ul>\n<li>Tianhe-2 ≈ <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>7.1</mn><mi>M</mi><mi mathvariant=\"normal\">/</mi><mi>P</mi><mi>F</mi><mo separator=\"true\">;</mo><mi>S</mi><mi>u</mi><mi>n</mi><mi>w</mi><mi>a</mi><mi>y</mi><mo>≈</mo></mrow><annotation encoding=\"application/x-tex\">7.1M/PF; Sunway ≈</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:1em;vertical-align:-0.25em;\"></span><span class=\"mord\">7.1</span><span class=\"mord mathnormal\" style=\"margin-right:0.109em;\">M</span><span class=\"mord\">/</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">P</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mpunct\">;</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord mathnormal\" style=\"margin-right:0.0576em;\">S</span><span class=\"mord mathnormal\">u</span><span class=\"mord mathnormal\">n</span><span class=\"mord mathnormal\" style=\"margin-right:0.0269em;\">w</span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mspace\" style=\"margin-right:0.2778em;\"></span><span class=\"mrel\">≈</span></span></span></span>2.2M/PF; Summit ≈ <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>1.0</mn><mi>M</mi><mi mathvariant=\"normal\">/</mi><mi>P</mi><mi>F</mi><mo separator=\"true\">;</mo><mi>F</mi><mi>u</mi><mi>g</mi><mi>a</mi><mi>k</mi><mi>u</mi><mo>≈</mo></mrow><annotation encoding=\"application/x-tex\">1.0M/PF; Fugaku ≈</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:1em;vertical-align:-0.25em;\"></span><span class=\"mord\">1.0</span><span class=\"mord mathnormal\" style=\"margin-right:0.109em;\">M</span><span class=\"mord\">/</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">P</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mpunct\">;</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">ug</span><span class=\"mord mathnormal\" style=\"margin-right:0.0315em;\">ak</span><span class=\"mord mathnormal\">u</span><span class=\"mspace\" style=\"margin-right:0.2778em;\"></span><span class=\"mrel\">≈</span></span></span></span>1.9M/PF (includes national R&amp;D); LUMI ≈ €0.4M/PF (lifecycle); Frontier ≈ <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>0.36</mn><mi>M</mi><mi mathvariant=\"normal\">/</mi><mi>P</mi><mi>F</mi><mo separator=\"true\">;</mo><mi>A</mi><mi>u</mi><mi>r</mi><mi>o</mi><mi>r</mi><mi>a</mi><mo>≈</mo></mrow><annotation encoding=\"application/x-tex\">0.36M/PF; Aurora ≈</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:1em;vertical-align:-0.25em;\"></span><span class=\"mord\">0.36</span><span class=\"mord mathnormal\" style=\"margin-right:0.109em;\">M</span><span class=\"mord\">/</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">P</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mpunct\">;</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord mathnormal\">A</span><span class=\"mord mathnormal\">u</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">or</span><span class=\"mord mathnormal\">a</span><span class=\"mspace\" style=\"margin-right:0.2778em;\"></span><span class=\"mrel\">≈</span></span></span></span>0.25M/PF; El Capitan ≈ $0.22M/PF. Trend: ~30× cheaper per FLOP in a decade, driven by GPU/AI silicon.</li>\n</ul>\n<p><strong>Effective <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mi mathvariant=\"normal\">/</mi><mi>F</mi><mi>L</mi><mi>O</mi><mi>P</mi><mo>−</mo><mi>h</mi><mi>o</mi><mi>u</mi><mi>r</mi><mo stretchy=\"false\">[</mo><mi>d</mi><mi>e</mi><mi>r</mi><mi>i</mi><mi>v</mi><mi>e</mi><mi>d</mi><mi>e</mi><mi>s</mi><mi>t</mi><mi>i</mi><mi>m</mi><mi>a</mi><mi>t</mi><mi>e</mi><mo separator=\"true\">,</mo><mi>e</mi><mi>l</mi><mi>e</mi><mi>c</mi><mi>t</mi><mi>r</mi><mi>i</mi><mi>c</mi><mi>i</mi><mi>t</mi><mi>y</mi><mi>o</mi><mi>n</mi><mi>l</mi><mi>y</mi><mo stretchy=\"false\">]</mo><mo>:</mo><mo>∗</mo><mo>∗</mo><mi>a</mi><mi>t</mi></mrow><annotation encoding=\"application/x-tex\">/FLOP-hour [derived estimate, electricity only]:** at</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:1em;vertical-align:-0.25em;\"></span><span class=\"mord\">/</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mord mathnormal\">L</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">O</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">P</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span><span class=\"mbin\">−</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:1em;vertical-align:-0.25em;\"></span><span class=\"mord mathnormal\">h</span><span class=\"mord mathnormal\">o</span><span class=\"mord mathnormal\">u</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mopen\">[</span><span class=\"mord mathnormal\">d</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">er</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">v</span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\">d</span><span class=\"mord mathnormal\">es</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\">ima</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\">e</span><span class=\"mpunct\">,</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\" style=\"margin-right:0.0197em;\">l</span><span class=\"mord mathnormal\">ec</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\">c</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mord mathnormal\">o</span><span class=\"mord mathnormal\">n</span><span class=\"mord mathnormal\" style=\"margin-right:0.0197em;\">l</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mclose\">]</span><span class=\"mspace\" style=\"margin-right:0.2778em;\"></span><span class=\"mrel\">:</span><span class=\"mspace\" style=\"margin-right:0.2778em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:0.4653em;\"></span><span class=\"mord\">∗</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span><span class=\"mbin\">∗</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:0.6151em;\"></span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\">t</span></span></span></span>0.05–0.10/kWh, Frontier&#39;s 21.1 MW costs ~<span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>1</mn><mo separator=\"true\">,</mo><mn>050</mn><mtext>–</mtext><mn>2</mn><mo separator=\"true\">,</mo><mn>100</mn><mi>p</mi><mi>e</mi><mi>r</mi><mi>m</mi><mi>a</mi><mi>c</mi><mi>h</mi><mi>i</mi><mi>n</mi><mi>e</mi><mo>−</mo><mi>h</mi><mi>o</mi><mi>u</mi><mi>r</mi><mo separator=\"true\">;</mo><mi>d</mi><mi>i</mi><mi>v</mi><mi>i</mi><mi>d</mi><mi>i</mi><mi>n</mi><mi>g</mi><mi>b</mi><mi>y</mi><mi>i</mi><mi>t</mi><mi>s</mi><mn>1.353</mn><mi>E</mi><mi>F</mi><mi>H</mi><mi>P</mi><mi>L</mi><mi>r</mi><mi>a</mi><mi>t</mi><mi>e</mi><mi>g</mi><mi>i</mi><mi>v</mi><mi>e</mi><mi>s</mi><mi>r</mi><mi>o</mi><mi>u</mi><mi>g</mi><mi>h</mi><mi>l</mi><mi>y</mi><mo>∗</mo><mo>∗</mo></mrow><annotation encoding=\"application/x-tex\">1,050–2,100 per machine-hour; dividing by its 1.353 EF HPL rate gives roughly **</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:0.8889em;vertical-align:-0.1944em;\"></span><span class=\"mord\">1</span><span class=\"mpunct\">,</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord\">050–2</span><span class=\"mpunct\">,</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord\">100</span><span class=\"mord mathnormal\">p</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">er</span><span class=\"mord mathnormal\">ma</span><span class=\"mord mathnormal\">c</span><span class=\"mord mathnormal\">hin</span><span class=\"mord mathnormal\">e</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span><span class=\"mbin\">−</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:0.8889em;vertical-align:-0.1944em;\"></span><span class=\"mord mathnormal\">h</span><span class=\"mord mathnormal\">o</span><span class=\"mord mathnormal\">u</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mpunct\">;</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord mathnormal\">d</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">v</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\">d</span><span class=\"mord mathnormal\">in</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">g</span><span class=\"mord mathnormal\">b</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\">s</span><span class=\"mord\">1.353</span><span class=\"mord mathnormal\" style=\"margin-right:0.0576em;\">E</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">F</span><span class=\"mord mathnormal\" style=\"margin-right:0.0813em;\">H</span><span class=\"mord mathnormal\" style=\"margin-right:0.1389em;\">P</span><span class=\"mord mathnormal\">L</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\">t</span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">g</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">v</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">esr</span><span class=\"mord mathnormal\">o</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">ug</span><span class=\"mord mathnormal\">h</span><span class=\"mord mathnormal\" style=\"margin-right:0.0197em;\">l</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span><span class=\"mbin\">∗</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:0.4653em;\"></span><span class=\"mord\">∗</span></span></span></span>800–1,600 per exaflop-hour</strong> of delivered Linpack work. Real applications sustain far below HPL, so effective cost per useful FLOP is typically 3–10× higher <strong>[rough estimate — electricity price and duty cycle assumptions, not an official figure]</strong>. Amortizing Frontier&#39;s <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>600</mn><mi>M</mi><mi>c</mi><mi>a</mi><mi>p</mi><mi>e</mi><mi>x</mi><mi>o</mi><mi>v</mi><mi>e</mi><mi>r</mi><mi>a</mi><mn>5</mn><mo>−</mo><mi>y</mi><mi>e</mi><mi>a</mi><mi>r</mi><mi>l</mi><mi>i</mi><mi>f</mi><mi>e</mi><mi>a</mi><mi>d</mi><mi>d</mi><mi>s</mi><mtext> </mtext></mrow><annotation encoding=\"application/x-tex\">600M capex over a 5-year life adds ~</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:0.8778em;vertical-align:-0.1944em;\"></span><span class=\"mord\">600</span><span class=\"mord mathnormal\" style=\"margin-right:0.109em;\">M</span><span class=\"mord mathnormal\">c</span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\">p</span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\">x</span><span class=\"mord mathnormal\">o</span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">v</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">er</span><span class=\"mord mathnormal\">a</span><span class=\"mord\">5</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span><span class=\"mbin\">−</span><span class=\"mspace\" style=\"margin-right:0.2222em;\"></span></span><span class=\"base\"><span class=\"strut\" style=\"height:0.8889em;vertical-align:-0.1944em;\"></span><span class=\"mord mathnormal\" style=\"margin-right:0.0359em;\">y</span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\" style=\"margin-right:0.0278em;\">r</span><span class=\"mord mathnormal\" style=\"margin-right:0.0197em;\">l</span><span class=\"mord mathnormal\">i</span><span class=\"mord mathnormal\" style=\"margin-right:0.1076em;\">f</span><span class=\"mord mathnormal\">e</span><span class=\"mord mathnormal\">a</span><span class=\"mord mathnormal\">dd</span><span class=\"mord mathnormal\">s</span><span class=\"mspace nobreak\"> </span></span></span></span>14k/machine-hour (~$10k per HPL-exaflop-hour) <strong>[derived]</strong>.</p>\n<p><strong>Access models:</strong></p>\n<ul>\n<li><strong>Open science (Frontier, Aurora, LUMI, Fugaku, Sunway-era OLCF):</strong> DOE splits LCF time ~<strong>60% INCITE / 30% ALCC / 10% Director&#39;s Discretionary</strong> (<a href=\"https://doeleadershipcomputing.org/wp-content/uploads/sites/123/2025/04/INCITE-Overview-and-Policies_2026.pdf\">INCITE Overview &amp; Policies</a>). INCITE is peer-reviewed and open to any affiliation; awards run to millions of node-hours. For 2026–27, ALCC pools: Frontier 20M node-hours (max request 2M), Aurora 16M (<a href=\"https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC/Allocation-Request-Cap\">ASCR ALCC cap page</a>).</li>\n<li><strong>LUMI:</strong> EuroHPC JU model — half the capacity for EU-wide calls, half for the hosting consortium countries (standard EuroHPC arrangement; <a href=\"https://lumi-supercomputer.eu/lumi-one-of-the-worlds-mightiest-supercomputers/\">LUMI/EuroHPC</a>) <strong>[standard model; not re-verified per-call]</strong>.</li>\n<li><strong>Fugaku:</strong> shared-use via RIKEN/HPCI since 9 Mar 2021 (<a href=\"https://global.fujitsu/en-global/technology/research/fugaku\">Fujitsu</a>).</li>\n<li><strong>Sierra, El Capitan:</strong> classified NNSA machines — no INCITE; tri-lab allocation via the ATCC process (<a href=\"https://www.osti.gov/servlets/purl/2475271\">ASC implementation plan</a>). El Capitan&#39;s unclassified sibling Tuolumne serves open science.</li>\n</ul>\n<hr>\n<h2 id=\"items-flagged-as-uncertain-or-unverifiable\">Items flagged as uncertain or unverifiable</h2><ul>\n<li>Sierra&#39;s standalone cost — never officially broken out of CORAL; the $525M figure covers the program.</li>\n<li>Fugaku&#39;s $1B is a &quot;reported&quot; figure (CACM); official MEXT budgets were spread over the ¥100B+ Flagship 2020 project.</li>\n<li>LUMI Rpeak appears as 531–552 PF depending on source; HPL 379.7 PF is solid.</li>\n<li>Summit&#39;s HPL power (<del>10 MW) vs full-system power (</del>13 MW) — sources conflate.</li>\n<li>The PPoPP 2022 &quot;10 billion atoms&quot; run&#39;s exact machine designation (described as the new-generation Sunway system; &quot;OceanLight&quot; naming is press usage).</li>\n<li>The 1.62-trillion-atom NEP result is a 2026 arXiv preprint, not yet peer-reviewed.</li>\n<li>All $/exaflop-hour numbers are my arithmetic on reported capex/power, not published values.</li>\n</ul>\n"}