\n\n⚠️ Stale / superseded summary. This is an extraction-process summary of the\nphonon report with dead
\n/sessions/...paths. For the current, complete review, see\ndocs/phonon_benchmarking_report.md.
The Phonon Frequency Spectrum Benchmarking Deep Research report provides a comprehensive technical framework for assessing 23 interatomic potentials across 12,000 materials (276,000 phonon calculations total). The report establishes systematic methodologies, performance metrics, and computational strategies essential for the GLIM project.
\nKey Insight: Phonon frequencies probe second-order energy derivatives, making them exponentially more sensitive to potential errors than energies/forces (which depend on zeroth/first derivatives).
\nGLIM Application: Phonon errors directly translate to property errors (~1% frequency error → ~1% entropy/free energy error, ~2% heat capacity error at room temperature). Explicit phonon testing prevents catastrophic failures in derived property predictions.
\nClear Performance Ranking:
\nTop Performers:
\nGLIM Implication: Pareto frontier analysis enables optimal potential selection based on accuracy vs. computational cost constraints.
\nSystematic Accuracy Pattern:\nMain-group compounds > Transition metals > Heavy elements/Rare earths
\nChallenging Systems:
\nGLIM Strategy: Stratified benchmarking by crystal system, chemistry (binary/ternary/quaternary), and bonding type enables identification of hidden failure modes and targeted improvements.
\nCritical Discovery: ORB and OMat show dramatic MAE increase at small displacements (attributed to direct force output vs. Hessian-based approaches).
\nImplication: Displacement magnitude selection (standard 0.01–0.03 Å) is architecture-dependent. Quality control requires validation of force constant behavior across displacement ranges.
\nJARVIS-DFT Database:
\nGLIM Consideration: Functional mismatch between PBE-trained models and optB88 reference creates systematic bias. Recommendation: functional-corrected metrics or materials-specific comparison subsets.
\nPhonon Force Constant Tuning (PFT):
\nGLIM Application: Fine-tuning pathways enable rapid accuracy optimization for top-performing potentials. The transfer from harmonic Hessian training to anharmonic properties suggests curvature supervision captures essential physics beyond harmonic regime.
\nKey Result: 25.83% baseline stability rate in AI-generated structures (high class imbalance).
\nMetric Interpretation:
\nGLIM Use Case: Potentials must reliably identify imaginary modes for structure screening in materials discovery pipelines.
\nEfficiency Metrics:
\nCost Estimation for GLIM:
\nOptimization Strategy:
\nPrimary Metrics (Global Ranking):
\nSecondary Metrics (Pattern Analysis):
\nDiagnostic Metrics (Improvement Guidance):
\nFramework Design:
\nUltimate Goal: Living benchmark that continuously improves with community contribution, enabling sustained progress in interatomic potential development.
\nAdopt stratified sampling: By crystal system, chemistry type, bonding character to ensure balanced coverage and reveal hidden failure modes.
\nImplement functional sensitivity analysis: Systematic PBE-PBEsol-optB88 comparisons on representative subsets to quantify functional bias.
\nPrioritize fine-tuning: Top performers benefit from Hessian-targeted training with modest computational investment.
\nMulti-level metric reporting: Provide frequency-level details (band structure), PDOS-level summaries, and property-level impacts simultaneously.
\nComposition-specific skill scores: Report baseline-corrected metrics stratified by chemistry to enable fair multi-material comparison.
\nDocumentation of failure modes: Catalog systematic error patterns (displacement-dependent, mode-specific, chemistry-specific) to guide potential development.
\nBenchmark versioning: Maintain backward compatibility while enabling evolving methodology through version tracking.
\nFull report: /sessions/friendly-gracious-hamilton/breadth_exploration/dr_reports/phonon_benchmarking_report.md