Direct prediction of phonon density of states with Euclidean neural networks
Zhantao Chen, Nina Andrejevic, Tess Smidt, Zhiwei Ding, Yen-Ting Chi, Quynh T. Nguyen, Ahmet Alatas, Jing Kong, Mingda Li
DDirect prediction of phonon density of states with Euclidean neural network
Zhantao Chen,
1, 2, ∗ Nina Andrejevic,
1, 3, ∗ Tess Smidt,
4, 5, ∗ Zhiwei Ding, Yen-TingChi, Quynh T. Nguyen, Ahmet Alatas, Jing Kong, and Mingda Li
1, 9, † Quantum Matter Group, MIT, Cambridge, MA 02139 Department of Mechanical Engineering, MIT, Cambridge, MA 02139 Department of Materials Science and Engineering, MIT, Cambridge, MA 02139 Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720 Center for Advanced Mathematics for Energy Research Applications,Lawrence Berkeley National Laboratory, Berkeley, CA 94720 Department of Physics, MIT, Cambridge, MA 02139 Advanced Photon Source, Argonne National Laboratory, Lemont IL 60439 Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA 02139 Department of Nuclear Science and Engineering, MIT, Cambridge, MA 02139 (Dated: September 14, 2020)Machine learning has demonstrated great power in materials design, discovery, and propertyprediction. However, despite the success of machine learning in predicting discrete properties,challenges remain for continuous property prediction. The challenge is aggravated in crystallinesolids due to crystallographic symmetry considerations and data scarcity. Here we demonstrate thedirect prediction of phonon density of states using only atomic species and positions as input. Weapply Euclidean neural networks, which by construction are equivariant to 3D rotations, translations,and inversion and thereby capture full crystal symmetry, and achieve high-quality prediction usinga small training set of ∼ examples with over 64 atom types. Our predictive model reproduceskey features of experimental data and even generalizes to materials with unseen elements. Wedemonstrate the potential of our network by predicting a broad number of high phononic specificheat capacity materials. Our work indicates an efficient approach to explore materials’ phononstructure, and can further enable rapid screening for high-performance thermal storage materialsand phonon-mediated superconductors. INTRODUCTION
One central objective of materials science is to estab-lish structure-property relationships; that is, how spe-cific atomic arrangements lead to certain macroscopicfunctionalities. This question is historically addressedthrough trial-and-error of a combination of structure andproperty characterization, theory, and calculation. How-ever, recent advances in machine learning (ML) suggesta paradigm shift in how structure-property relationshipscan be directly constructed [1, 2]. To date, ML hasseen success in a growing spectrum of materials appli-cations, including materials discovery and design [3–6],process automation and optimization [7, 8], and pre-diction of materials’ mechanical (elastic moduli) [9–12],thermodynamic and thermal transport (formation en-thalpy, thermal conductivity, Debye temperature, heatcapacity) [10, 12–16], and electronic (bandgap, supercon-ductivity, topology) properties [11, 17–23], and atomisticpotentials (potential energy surfaces and force constants)[24–30]. Most property prediction studies consider alow-dimensional output consisting of one or few discretepoints. However, the prediction of continuous propertiesfrom limited input information remains challenging dueto the output complexity and finite data volume. More-over, for crystalline solids, the crystallographic symmetryposes additional constraints on a generic neural network.In this work, we build a ML-based predictive model that directly outputs the phonon density of states (DoS)using atomic structures as input. Phonon DoS is akey determinant of materials’ specific heat and vibra-tional entropy and plays a crucial role in interfacial ther-mal resistance [31]. It is also tightly linked to thermaland electrical transport [32] and superconductivity [33].However, the acquisition of experimental and computedphonon DoS is nontrivial due to limited inelastic scatter-ing facility resources and high computational cost of abinitio calculations for complex materials [32, 34]. Thiscalls for an approach that acquires phonon DoS more ef-ficiently. To build such a model, we employ a Euclideanneural network (E(3)NN) which naturally operates on3D geometry and is equivariant to 3D translations, rota-tions, and inversion [35–38]. E(3)NNs preserve all geo-metric information of the input and eliminate the needfor expensive (approximately 500 fold) data augmenta-tion. Additionally, all crystallographic symmetries ofinput data are preserved by the network [39]. In thiswork, we use E(3)NNs as implemented in the open-source e3nn repository [38] which merges implementations ofRef. [35] and Ref. [37] and additionally implements inver-sion symmetry. High-fidelity phonon DoS predictions areachieved using the density functional perturbation theory(DFPT)-based phonon database [40] containing phononDoS data of approximately 1,500 crystalline solids. Ourpredictive model can capture the main features of phononDoS, even for crystalline solids with unseen elements. By a r X i v : . [ phy s i c s . c o m p - ph ] S e p Figure 1. Overview of the E(3)NN architecture for phonon DoS prediction. (a) Crystals are converted to periodic graphs byconsidering all periodic neighbors within a radial cutoff r max = 5 Å. The example of SrTiO is shown. (b) Atom types areencoded as a mass-weighted one-hot encoding. (c) Edges join neighboring atoms and store the relative distance vector from thecentral atom to neighbor. (d) The radial distance vectors are used for the continuous convolutional filters W ( (cid:126)r ab ) comprisinglearned radial functions and spherical harmonics. (e) The E(3)NN operates on the node and edge features using convolutionand gated nonlinear layers. The result is passed to a final activation, aggregation, and normalization to generate the predictedoutput. The network weights are trained by minimizing the loss function between the predicted and ground-truth phonon DoS. predicting the phonon DoS in 4,346 new crystal struc-tures, we identify a list of high heat capacity materi-als, supported by additional DFPT calculations. Ourwork offers an efficient technique to acquire phonon DoSdirectly from atomic structure, making it suitable forhigh throughput materials design with desirable phonon-related properties. EUCLIDEAN NEURAL NETWORK FORPHONON DOS PREDICTION
Crystal structures operated on by the E(3)NN are firstconverted into a periodic graphs where atoms are nodes N with edges E connecting neighboring atoms withina specified radial cutoff, including periodic images (Fig-ure 1). Each edge e ab ∈ E stores the radial distancevector between atom a and neighbor b , (cid:126)r ab , up to someradial cutoff | r max | , and is used by the convolutional ker-nels of the E(3)NN. The input node features are scalarsthat captures its atomic type and mass using one-hotencoding; for instance, a hydrogen atom is encoded as x H = [ m H , , . . . , . After an initial embedding layerwhich takes the 118-length one-hot mass-weighted en-codings to 64 scalar features, the constructed graph isthen passed to the E(3)NN, which iteratively operateson the features with multiple “Convolution and GatedBlock” layers as described for the L1Net of Ref.[41] (seeSupplementary Material for more details). After the finallayer, which consists of only a convolution, all resultingnode features are summed and passed through a final ac- tivation (ReLU) and normalization layer to predict thephonon DoS, comprising 51 scalars. The absolute mag-nitude of the phonon DoS can easily be recovered fromthe normalized DoS by noticing that (cid:82) g ( ω )d ω = 3 N ,where N is the number of atoms in the unit cell; thus,we ensure that normalization of the DoS does not com-promise meaningful prediction. The E(3)NN weights areoptimized by minimizing the mean squared error (MSE)loss function between the DFPT-computed DoS g andE(3)NN-predicted ˆ g . The full network structure is pro-vided in the Supplementary Material.We perform several analyses to evaluate our modelgiven the limited training data. Figure 2a shows thatthere is no obvious correlation between the MSE and thenumber of basis atoms within unit cells among training,validation, and test datasets (additional statistics in areavailable in the Supplementary Material). The overalltest set error is higher compared to the training set butsimilar to that of the validation set, suggesting good gen-eralizability. We also present the MSE as a function ofdifferent elements (Figure 2b) and observe comparableerror levels, indicating balanced prediction. Lastly, wecompute the average phonon frequency ¯ ω = (cid:82) d ω g ( ω ) ω (cid:82) d ω g ( ω ) forboth E(3)NN-predicted and DFPT ground-truth spectra,which show excellent agreement on the test set (Figure2c); specifically, for 70% of the testing samples, the rel-ative error is below 10%. This strongly suggests the ca-pability of our model to predict phonon DoS.To visualize the model performance, we plot 7 ran-domly selected examples from the test set in each error NaN Cd(InSe ) Zn(InTe ) CsSrBr Sr SbAu TlBr SrCl Sr(SbO ) BaHfN K PtSe Rb SeCl Cd(AsO ) Cs TiCl Sr(ZnP) KRb InCl LiI KSbO BaTeO NaAuO CaSO ZnSnP NaPF CdCN NaCNO RuS Li GeO CaHCl P Pt Number of Sites ° ° M ean S qua r ed E rr o r (a) (b) M ean S qua r ed E rr o r Ag Al As Au B Ba Be Bi Br C Ca Cd Cl Co Cr Cs Cu F Fe Ga Ge H Hf Hg I In Ir K La Li Mg Mn ° ° ° Train Validation Test
Mo N Na Nb Ni O Os P Pb Pd Pt Rb Re Rh Ru S Sb Sc Se Si Sn Sr Ta Tc Te Ti Tl V W Y Zn Zr ° ° ° True ¯ ! § (cm ° ) P r ed i c t ed ¯ ! ( c m ° ) R e l a t i v e E rr o r (c)(d) Figure 2. Performance of the Euclidean neural network-based predictive model. (a) Mean squared error versus total number ofsites in a unit cell in training (blue), validation (orange), and test (green) sets. (b) Average mean squared error of compoundscontaining each element. (c) Comparison between E(3)NN-predicted average phonon frequency and ground truth. The insetshows the relative error | ¯ ω − ¯ ω ∗ | / ¯ ω ∗ distribution of the three datasets. (d) Randomly selected examples in the test set withineach error quartile. (Left) MSE distribution showing that it is heavily peaked in the 1 st and 2 nd quartiles with lower error. quartile in Figure 2d, with rows 1 through 4 correspond-ing to the 1 st quartile with highest agreement throughthe 4 th quartile with lowest agreement, respectively. Ad-ditional examples are plotted in the Supplementary Ma-terial. The predicted DoS in the 1 st and 2 nd quartilesshow excellent agreement with DFPT calculations by re-producing fine features, while the 3 rd and 4 th quantilesshow good or acceptable agreement by capturing mainfeatures. For instance, the predicted DoS of NaPF ,CdCN , and NaCNO capture the energy of acoustic andoptical phonon branches well but mispredict the relativeamplitudes of certain peaks. Nonetheless, the phononbandgap, a key quantity to determine phonon-phononscattering, can be accurately extracted. Similarly, forKSbO and Li GeO , the predictions exhibit broadbandDoS distribution, agreeing with DFPT calculations. Alarge discrepancy can be seen for RuS , yet the band- width agreement is still good although the 0.099 MSEis among the largest errors in the test set (Figure 2a).The good test set performance and generalizability sug-gest the suitability of our model to predict phonon DoSfor a broad range of new materials. COMPARISON TO EXPERIMENTAL DATA
To demonstrate our model’s predictive ability in cap-turing experimental characteristics, we compare theE(3)NN predictions in a few materials with experimentalDoS data available from inelastic scattering (Figure 3).Although the E(3)NN-predicted DoS do not match thefine features of the experimental spectra, several key fea-tures, like peak positions, gaps, and energy bandwidths,are still well-reproduced. It is worth mentioning thatour model was not trained on any materials containing
Frequency (cm − ) P honon D o S ( a . u . ) ZrSiO SrTiO Nd CuO Sb Te UO YBa Cu O Figure 3. Comparison between E(3)NN model predictions(orange curves) and inelastic scattering DoS data (blue dots),reproduced from literature [42–48]. ZrSiO was in the train-ing set and SrTiO in the test set (black dashed lines denotecorresponding DFPT results [40]). The remaining exampleswere absent in all datasets used for training, validation, andtesting, and contain two unseen elements, Nd and U. uranium or neodymium, but still yields good DoS pre-dictions for UO and Nd CuO with general peak agree-ment. Given the disorder and anharmonic effects in ameasured sample, disagreement between DFPT calcu-lations and measured data can happen. As a result,lower agreement is expected between experimental andE(3)NN-predicted DoS since the ground-truths are basedon DFPT calculations. However, the predictive ability onmain features like peak positions, gaps, and bandwidthscan still serve as useful guidance for planning inelasticneutron and x-ray scattering measurements, where ex-perimental resources are largely limited to national lab-oratory facilities. HIGH-THROUGHPUT PHONON DOS ANDSPECIFIC HEAT CAPACITY PREDICTIONS
We apply the predictive model on 4,346 unseen crystalstructures without ground-truth DoS from the MaterialsProject [49] with atomic site number N ≤ in each unitcell. As a check, we plot the average phonon frequency ¯ ω against the average atomic mass ¯ m = ( N (cid:80) Ni =1 √ m i ) inthe unit cell (Figure 4a). The data fit well to a hyperboliccurve ¯ ω = C ¯ m − / , where the constant C is a measure ofthe crystal rigidity. The reasonable distribution of rigid-ity supports the physical validity of our model for newmaterials. Moreover, we characterize the non-uniformityof atomic masses in each material by computing the ra-tio of the minimum mass m min in a crystal to ¯ m , wherehigh m min / ¯ m tends to aggregate at lower ¯ ω and higher ¯ m (Figure 4b), which agrees with the features in [40].Figure 5a illustrates the average phononic specific heatcapacity C V of crystalline solids containing a given ele- m (amu) ¯ ! ( c m ° ) m min / ¯ m (a) m min / ¯ m ¯ ! ( c m ° ) ¯ m (b) Figure 4. Evaluation of model predictions on unseen crystalstructures. (a) Average frequency ¯ ω versus average atomicmass ¯ m . The black solid line represents the least squared fitto the hyperbolic relation ¯ ω ∼ ¯ m − / . (b) The average fre-quency versus the ratio m min / ¯ m characterizing atomic massnon-uniformity. ment, using the relation [50] C V ( T ) = k B m tot (cid:90) ∞ (cid:16) (cid:126) ω k B T (cid:17) csch (cid:16) (cid:126) ω k B T (cid:17) g ( ω ) dω, (1)where m tot is the total mass of all N atoms in theunit cell, and the phonon DoS is normalized such that (cid:82) g ( ω ) dω = 3 N . Materials containing light elements tendto have high heat capacity, which is reasonable. The dis-tribution of C V evaluated from Eq. 1 is shown in Figure5b, where ∼ of materials show a C V greater than1,000 J / (kg · K) . The inset shows the average phononDoS of highest- C V materials. Materials with higher C V appear to have high spectral weight at higher energies,consistent with expectation. This trend is also noticed byinspecting the scatter plot of phonon DoS along the firsttwo principal components (Figure 5c), where high heatcapacity materials appear clustered with respect to thefirst principal axis. The first principal axis has a broadnegative peak extending to high energies; thus, the clus-tering of high C V materials in the negative first principaldirection parallels the shift of their phonon DoS towardshigher energies.To validate our model’s predictions of high C V ma-terials, we select 12 materials with ultrahigh predicted C V and carry out independent DFPT calculations. Sincethe maximum frequency of the training data was set to1,000 cm − , which is sufficient for the majority of materi-als, the C V evaluated by DFPT was also cut off at 1,000cm − for fair comparison. The DoS comparisons betweenE(3)NN and DFPT in these high- C V materials are shownin 5d, where satisfactory agreements are achieved in mostexamples except for H NF and HCN. The C V at roomtemperature T = 293 .
15 K evaluated from E(3)NN pre-dictions and DFPT calculations are summarized in TableI and plotted in Figure 5e, showing excellent agreement.A full energy range comparison is presented in the Sup-plementary Material. We attribute the large discrepancy
Frequency (cm ° ) P honon D O S ( a . u . ) LiMgN, LiC, LiBO ,LiBeBO , MgC , MgH ,Li S N, Li Al , H NF, Li HN,
HCN, Li Al , (a) (d) (b) (c) Calculated C v P r ed i c t ed C v (e) ° r i n c i pa l c o m ponen t C v P r i n c i pa l a x e s Heat Capacity C v ( J / (kg · K) ) C oun t s Frequency (cm ° ) P honon D O S ( a . u . ) Top 5%Top 50%Top 75% C v Figure 5. The search for high specific heat capacity ( C V ) materials. (a) Periodic table colored by average C V of materialscontaining each element. (b) Histogram showing distribution of C V evaluated from E(3)NN-predicted phonon DoS. The insetillustrates average phonon DoS of materials with highest C V . (c) Distribution of predicted phonon DoS along the first twoprincipal components, colored by the E(3)NN-predicted C V magnitudes. The inset shows first two principal axes in the originalfrequency basis. (d) Comparison between E(3)NN-predicted and DFPT-computed phonon DoS, the dashed black curvesrepresent DFPT results. (e) Two-dimensional histogram comparing specific heat capacities evaluated from E(3)NN-predictedand DFPT-calculated phonon DoS.Table I. Comparisons of E(3)NN predicted and DFPT calcu-lated specific heat capacities (in J / (kg · K) ).Material C E(3)NN V C DFPT V Material C E(3)NN V C DFPT V Li HN 2514.9 2320.2 Li S N 1508.1 1498.3H NF 2396.6 3117.4 LiBeBO Al Al in hydrogen- and lithium-rich materials to the electro-static effect of hydrogen and lithium bonding [51], whilethe current model mainly considers the mass effect. DISCUSSION
In this work, we present a machine learning-based pre-dictive model to directly acquire the high-dimensional material property of phonon DoS, using only “first-principles” inputs, namely atomic species and positions.Due to their equivariance, Euclidean neural networks areable to capture the symmetries of the input crystal, mak-ing them data-efficient. A small training set of only 1,200examples is sufficient to generate meaningful predictions,outperforming a well-trained convolutional neural net-work even with data augmentation (Supplementary Ma-terial). Euclidean neural networks can be applied topredicting broader properties in crystalline solids, wherethere are often issues of data scarcity.In contrast to ab initio calculations and inelastic scat-tering that acquire phonon DoS deterministically, a MLmodel is data-driven and probabilistic in nature. It isthus impractical to fully rely on a ML-based predictivemodel to acquire materials properties without furthervalidation. However, the power of the ML approachgoes far beyond obtaining property magnitudes at theindividual material level. From a materials design per-spective, ML demonstrates extremely high efficiency inrapidly screening candidates with a target property. Inour case, the prediction of phonon DoS on the 4,346 un-seen materials can be done in less than 30 minutes ona single entry-level GPU. From a property optimizationperspective, instead of measuring an individual mate-rial with high precision, the ML approach searches out-puts high-performance candidates in a batch, offeringmore choices. From an experimental perspective, the MLmodel can be valuable in guiding the experimental plan-ning with limited national facility resources. From anapplication point-of-view, the Dulong-Petit law poses agrand challenge in searching for promising materials forthermal storage [52], and a highly efficient approach canfurther support an inverse design from property to desir-able structures. In sum, our model provides a promis-ing framework to enable high-throughput screening andguide experimental planning for materials with excep-tional thermal properties. It further sheds light on eluci-dating the fundamental links between symmetry, struc-ture, and elementary excitations in condensed matter.
ACKNOWLEDGMENTS
Z.C., N.A. and M.L. thank C.H. Rycroft and J.W.Lynn for helpful discussions. Z.C., N.A., and M.L. ac-knowledge the support from U.S. DOE, Office of Sci-ence (SC), Basic Energy Sciences (BES), award No. DE-SC0020148. N.A. acknowledges the support of the Na-tional Science Foundation Graduate Research FellowshipProgram under Grant No. 1122374. Y.T. thanks K. VanVliet for computational support. ∗ These authors contributed equally to this work. † Corresponding [email protected][1] R. Ramprasad, R. Batra, G. Pilania, A. Mannodi-Kanakkithodi, and C. Kim, “Machine learning in mate-rials informatics: recent applications and prospects,” npjComputational Materials , vol. 3, no. 1, pp. 1–13, 2017.[2] J. Schmidt, M. R. Marques, S. Botti, and M. A. Marques,“Recent advances and applications of machine learning insolid-state materials science,” npj Computational Mate-rials , vol. 5, no. 1, pp. 1–36, 2019.[3] P. Raccuglia, K. C. Elbert, P. D. Adler, C. Falk, M. B.Wenny, A. Mollo, M. Zeller, S. A. Friedler, J. Schrier,and A. J. Norquist, “Machine-learning-assisted materi-als discovery using failed experiments,”
Nature , vol. 533,no. 7601, pp. 73–76, 2016.[4] A. O. Oliynyk, E. Antono, T. D. Sparks, L. Ghad-beigi, M. W. Gaultois, B. Meredig, and A. Mar, “High-throughput machine-learning-driven synthesis of full-heusler compounds,”
Chemistry of Materials , vol. 28,no. 20, pp. 7324–7331, 2016.[5] R. Gómez-Bombarelli, J. N. Wei, D. Duvenaud, J. M.Hernández-Lobato, B. Sánchez-Lengeling, D. Sheberla,J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams,and A. Aspuru-Guzik, “Automatic chemical design using a data-driven continuous representation of molecules,”
ACS central science , vol. 4, no. 2, pp. 268–276, 2018.[6] Y. Liu, T. Zhao, W. Ju, and S. Shi, “Materials discoveryand design using machine learning,”
Journal of Materi-omics , vol. 3, no. 3, pp. 159 – 177, 2017. High-throughputExperimental and Modeling Research toward AdvancedBatteries.[7] F. Häse, L. M. Roch, and A. Aspuru-Guzik, “Chimera:enabling hierarchy based multi-objective optimization forself-driving laboratories,”
Chem. Sci. , vol. 9, pp. 7642–7655, 2018.[8] J. M. Granda, L. Donina, V. Dragone, D.-L. Long, andL. Cronin, “Controlling an organic synthesis robot withmachine learning to search for new reactivity,”
Nature ,vol. 559, pp. 377–381, Jul 2018.[9] G. Pilania, C. Wang, X. Jiang, S. Rajasekaran, andR. Ramprasad, “Accelerating materials property predic-tions using machine learning,”
Scientific reports , vol. 3,no. 1, pp. 1–6, 2013.[10] O. Isayev, C. Oses, C. Toher, E. Gossett, S. Curtarolo,and A. Tropsha, “Universal fragment descriptors for pre-dicting properties of inorganic crystals,”
Nature commu-nications , vol. 8, no. 1, pp. 1–12, 2017.[11] T. Xie and J. C. Grossman, “Crystal graph convolutionalneural networks for an accurate and interpretable pre-diction of material properties,”
Physical review letters ,vol. 120, no. 14, p. 145301, 2018.[12] C. Chen, W. Ye, Y. Zuo, C. Zheng, and S. P. Ong, “Graphnetworks as a universal machine learning framework formolecules and crystals,”
Chemistry of Materials , vol. 31,no. 9, pp. 3564–3572, 2019.[13] J. Carrete, W. Li, N. Mingo, S. Wang, and S. Curtarolo,“Finding unprecedentedly low-thermal-conductivity half-heusler semiconductors via high-throughput materialsmodeling,”
Physical Review X , vol. 4, no. 1, p. 011019,2014.[14] A. van Roekeghem, J. Carrete, C. Oses, S. Curtarolo,and N. Mingo, “High-throughput computation of thermalconductivity of high-temperature solid phases: the caseof oxide and fluoride perovskites,”
Physical Review X ,vol. 6, no. 4, p. 041061, 2016.[15] S. A. Tawfik, O. Isayev, M. J. Spencer, and D. A. Win-kler, “Predicting thermal properties of crystals using ma-chine learning,”
Advanced Theory and Simulations , vol. 3,no. 2, p. 1900208, 2020.[16] B. Mortazavi, E. V. Podryabinkin, S. Roche, T. Rabczuk,X. Zhuang, and A. V. Shapeev, “Machine-learninginteratomic potentials enable first-principles multi-scale modeling of lattice thermal conductivity ingraphene/borophene heterostructures,”
Materials Hori-zons , 2020.[17] L. Ward, A. Agrawal, A. Choudhary, and C. Wolver-ton, “A general-purpose machine learning framework forpredicting properties of inorganic materials,” npj Com-putational Materials , vol. 2, no. 1, pp. 1–7, 2016.[18] Y. Zhuo, A. Mansouri Tehrani, and J. Brgoch, “Predict-ing the band gaps of inorganic solids by machine learn-ing,”
The journal of physical chemistry letters , vol. 9,no. 7, pp. 1668–1673, 2018.[19] Y. Dong, C. Wu, C. Zhang, Y. Liu, J. Cheng, andJ. Lin, “Bandgap prediction by deep learning in config-urationally hybridized graphene and boron nitride,” npjComputational Materials , vol. 5, no. 1, pp. 1–8, 2019. [20] B. Meredig, E. Antono, C. Church, M. Hutchinson,J. Ling, S. Paradiso, B. Blaiszik, I. Foster, B. Gibbons,J. Hattrick-Simpers, et al. , “Can machine learning iden-tify the next high-temperature superconductor? exam-ining extrapolation performance for materials discovery,”
Molecular Systems Design & Engineering , vol. 3, no. 5,pp. 819–825, 2018.[21] V. Stanev, C. Oses, A. G. Kusne, E. Rodriguez,J. Paglione, S. Curtarolo, and I. Takeuchi, “Machinelearning modeling of superconducting critical tempera-ture,” npj Computational Materials , vol. 4, no. 1, pp. 1–14, 2018.[22] N. Andrejevic, J. Andrejevic, C. H. Rycroft, and M. Li,“Machine learning spectral indicators of topology,” arXivpreprint arXiv:2003.00994 , 2020.[23] N. Claussen, B. A. Bernevig, and N. Regnault, “Detectionof topological materials with machine learning,”
PhysicalReview B , vol. 101, no. 24, p. 245117, 2020.[24] Z. Li, J. R. Kermode, and A. De Vita, “Molecular dy-namics with on-the-fly machine learning of quantum-mechanical forces,”
Physical review letters , vol. 114, no. 9,p. 096405, 2015.[25] S. Chmiela, A. Tkatchenko, H. E. Sauceda, I. Poltavsky,K. T. Schütt, and K.-R. Müller, “Machine learning of ac-curate energy-conserving molecular force fields,”
Scienceadvances , vol. 3, no. 5, p. e1603015, 2017.[26] I. Kruglov, O. Sergeev, A. Yanilkin, and A. R. Oganov,“Energy-free machine learning force field for aluminum,”
Scientific reports , vol. 7, no. 1, pp. 1–7, 2017.[27] A. Glielmo, P. Sollich, and A. De Vita, “Accurate inter-atomic force fields via machine learning with covariantkernels,”
Physical Review B , vol. 95, no. 21, p. 214302,2017.[28] V. Botu, R. Batra, J. Chapman, and R. Ramprasad, “Ma-chine learning force fields: construction, validation, andoutlook,”
The Journal of Physical Chemistry C , vol. 121,no. 1, pp. 511–522, 2017.[29] F. Legrain, A. van Roekeghem, S. Curtarolo, J. Carrete,G. K. Madsen, and N. Mingo, “Vibrational properties ofmetastable polymorph structures by machine learning,”
Journal of Chemical Information and Modeling , vol. 58,no. 12, pp. 2460–2466, 2018.[30] L. Zhang, D.-Y. Lin, H. Wang, R. Car, and E. Weinan,“Active learning of uniformly accurate interatomic poten-tials for materials simulation,”
Physical Review Materi-als , vol. 3, no. 2, p. 023804, 2019.[31] Y.-J. Wu, L. Fang, and Y. Xu, “Predicting interfacialthermal resistance by machine learning,” npj Computa-tional Materials , vol. 5, no. 1, p. 56, 2019.[32] F. Giustino, “Electron-phonon interactions from firstprinciples,”
Rev. Mod. Phys. , vol. 89, p. 015003, Feb 2017.[33] J. Bardeen, L. N. Cooper, and J. R. Schrieffer, “Theoryof superconductivity,”
Physical review , vol. 108, no. 5,p. 1175, 1957.[34] S. Baroni, S. De Gironcoli, A. Dal Corso, and P. Gi-annozzi, “Phonons and related crystal properties fromdensity-functional perturbation theory,”
Reviews of Mod-ern Physics , vol. 73, no. 2, p. 515, 2001.[35] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li,K. Kohlhoff, and P. Riley, “Tensor field networks:Rotation-and translation-equivariant neural networks for3d point clouds,” arXiv preprint arXiv:1802.08219 , 2018.[36] R. Kondor, Z. Lin, and S. Trivedi, “Clebsch–gordan nets:a fully fourier space spherical convolutional neural net- work,” in
Advances in Neural Information ProcessingSystems 32 , pp. 10117–10126, 2018.[37] M. Weiler, M. Geiger, M. Welling, W. Boomsma, andT. Cohen, “3D Steerable CNNs: Learning RotationallyEquivariant Features in Volumetric Data,” in
Advancesin Neural Information Processing Systems 32 , pp. 10402–10413, 2018.[38] M. Geiger, T. Smidt, B. K. Miller, W. Boomsma,K. Lapchevskyi, M. Weiler, M. Tyszkiewicz, andJ. Frellsen, “github.com/e3nn/e3nn,” May 2020.[39] T. E. Smidt, M. Geiger, and B. K. Miller, “Finding Sym-metry Breaking Order Parameters with Euclidean Neu-ral Networks,” arXiv e-prints , p. arXiv:2007.02005, July2020.[40] G. Petretto, S. Dwaraknath, H. P. Miranda, D. Winston,M. Giantomassi, M. J. Van Setten, X. Gonze, K. A. Pers-son, G. Hautier, and G.-M. Rignanese, “High-throughputdensity-functional perturbation theory phonons for inor-ganic materials,”
Scientific data , vol. 5, p. 180065, 2018.[41] B. K. Miller, M. Geiger, T. E. Smidt, and F. Noé, “Rel-evance of rotationally equivariant convolutions for pre-dicting molecular properties,” 2020.[42] N. Choudhury, E. J. Walter, A. I. Kolesnikov, and C.-K. Loong, “Large phonon band gap in sr ti o 3 and thevibrational signatures of ferroelectricity in a ti o 3 per-ovskites: First-principles lattice dynamics and inelasticneutron scattering,”
Physical Review B , vol. 77, no. 13,p. 134111, 2008.[43] J. Lynn, I. Sumarlin, D. Neumann, J. Rush, J. Peng, andZ. Li, “Phonon density of states and superconductivityin nd 1.85 ce 0.15 cuo 4,”
Physical review letters , vol. 66,no. 7, p. 919, 1991.[44] I. Sumarlin, J. Lynn, D. Neumann, J. Rush, C. Loong,J. Peng, and Z. Li, “Phonon density of states in r 2 cuo4 and superconducting r 1.85 ce 0.15 cuo 4 (r= nd, pr),”
Physical Review B , vol. 48, no. 1, p. 473, 1993.[45] H. Rauh, R. Geick, H. Kohler, N. Nucker, and N. Lehner,“Generalized phonon density of states of the layer com-pounds bi2se3, bi2te3, sb2te3 and bi2 (te0. 5se0. 5) 3,(bi0.5sb0. 5) 2te3,”
Journal of Physics C: Solid State Physics ,vol. 14, no. 20, p. 2705, 1981.[46] J. W. Pang, A. Chernatynskiy, B. C. Larson, W. J. Buy-ers, D. L. Abernathy, K. J. McClellan, and S. R. Phillpot,“Phonon density of states and anharmonicity of uo 2,”
Physical Review B , vol. 89, no. 11, p. 115132, 2014.[47] S. Chaplot, L. Pintschovius, N. Choudhury, and R. Mit-tal, “Phonon dispersion relations, phase transitions, andthermodynamic properties of zr si o 4: Inelastic neutronscattering experiments, shell model, and first-principlescalculations,”
Physical Review B , vol. 73, no. 9, p. 094308,2006.[48] J. Nipko and C.-K. Loong, “Inelastic neutron scatter-ing from zircon,”
Physica B: Condensed Matter , vol. 241,pp. 415–417, 1997.[49] A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards,S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, et al. , “Commentary: The materials project: A materialsgenome approach to accelerating materials innovation,”
Apl Materials , vol. 1, no. 1, p. 011002, 2013.[50] C. Lee and X. Gonze, “Ab initio calculation of the ther-modynamic properties and atomic temperature factors ofsio 2 α -quartz and stishovite,” Physical Review B , vol. 51,no. 13, p. 8610, 1995. [51] A. Shahi and E. Arunan, “Hydrogen bonding, halogenbonding and lithium bonding: an atoms in molecules andnatural bond orbital perspective towards conservationof total bond order, inter-and intra-molecular bonding,”
Physical Chemistry Chemical Physics , vol. 16, no. 42,pp. 22935–22952, 2014.[52] A. Henry, R. Prasher, and A. Majumdar, “Five thermalenergy grand challenges for decarbonization,”
Nature En-ergy , pp. 1–3, 2020.[53] S. P. Ong, W. D. Richards, A. Jain, G. Hautier,M. Kocher, S. Cholia, D. Gunter, V. L. Chevrier, K. A. Persson, and G. Ceder, “Python materials genomics (py-matgen): A robust, open-source python library for mate-rials analysis,”
Computational Materials Science , vol. 68,pp. 314–319, Feb. 2013.[54] J. Lynn, H. Smith, and R. Nicklow, “Lattice dynamics ofgold,”
Physical Review B , vol. 8, no. 8, p. 3493, 1973.[55] J. Hoffmann, L. Maestrati, Y. Sawada, J. Tang, J. M.Sellier, and Y. Bengio, “Data-driven approach to encod-ing and decoding 3-d crystal structures,” arXiv preprintarXiv:1909.00949 , 2019.
Supplementary Material
EUCLIDEAN NEURAL NETWORK IN DETAIL
As discussed in the main text, crystals are represented to the network as periodic graphs where relative distancevectors are edge attributes and atom features are stored at each node as shown in Figure 1. Crystal structuresare converted into graphs using the e3nn DataPeriodicNeighbors class which uses the pymatgen Structure classto find atom neighbors while accounting for periodic boundary conditions [53]. For each atom, all atoms around agiven atom within a given cutoff radius (including the central atom) are considered “neighbors” and are used in theconvolution operation. The features of each atom in the unit cell is stored in x ai where a is the atom index and i is aflattened representation index with features given by a representation list Rs = [(N, 0, 1)] which denotes N scalars.Representation lists in e3nn denote irreducible representations (irreps) of the group of 3D rotations and inversion, O (3) , and are lists of triples (m, L , p) denoting the number of copies or multiplicity m of features with rotationdegree L and parity p (1 for even parity, -1 for odd parity, and 0 for irreps of the group of just 3D rotations SO (3) .An irrep or rotation degree L has L + 1 components. By flattening the representations into one index and usingrepresentation lists, we can efficient store and operate on geometric tensor quantities expressed in the irrep basis.We employ the modified one-hot encoding for element types with atomic mass as magnitudes, namely V (0) vc | c = Z v = m v for site v with atomic number Z v and mass m v (note the subscript m ≡ is omitted for order l = 0 ). For example,H: [1 , , . . . , , , He: [0 , , . . . , , , . . . , Og: [0 , , . . . , , . (S1)The encoded crystal graph is then passed into a E(3)NN as illustrated in Figure S1. The architecture of the Euclideanneural networks used in this work is similar to that of a graph convolutional neural networks. To achieve Euclideansymmetry equivariance: 1) E(3)NN convolutional filters are functions of the radial distance vector between two pointsand composed of learned radial functions and spherical harmonics W ( (cid:126)r ) = R ( | r | ) Y lm (ˆ r ) . 2) As a consequence of thisfilter choice, all inputs, intermediate data, and outputs are geometric tensors. 3) Therefore, all scalar operations (e.g.addition and multiplication) in the network must be replaced with general geometric tensor algebra. 4) Additionally,nonlinearities applied to geometric tensor data must also be replaced with equivariant equivalents. A feature thatemerges from equivariance is that the symmetry of the outputs of Euclidean neural networks are guaranteed to haveequal or higher symmetry than the inputs, which means that these networks are guaranteed to respect the spacegroup symmetries (which are a subgroup of Euclidean symmetry) of input crystal geometry [39].To articulate network operations, we will use Einstein summation notation where repeated indices are implicitlysummed over. A single layer of our network operates on input x ai and relative distance vectors of graph edges (cid:126)r ab , x ( q +1) ai = σ ( x ( q ) bj ⊗ K k ( (cid:126)r ab )) (S2)where σ is an equivariant nonlinearity and ⊗ signifies a tensor product where representation indices of inputsand filter are contracted using Clebsch-Gordan coefficients. We used a “gated” rotation equivariant nonlinearity, GatedBlockParity as implemented in e3nn , which was first introduced in Ref. [37] and is extended in e3nn to handleparity (inversion). K is the convolutional kernel which is composed of learned radial functions and spherical harmonicsand Clebsch-Gordan coefficients are included in the kernel to yield the traditional channel out and in indices. K ij ( (cid:126)r ab ) = K abij = R w ( r ab ) Y k (ˆ r ab ) C ijk δ w,k ∈ irrep ( w ) (S3)where δ w,k ∈ irrep ( w ) denotes that radial functions are shared for all components of a given irrep, e.g. the 5 componentsof a L = 2 irrep share the same radial function, and C ijk are the Clebsch-Gordan coeffcients.In tensor notation, a convolutional operation is written asConv ( x bi , (cid:126)r ab ) := x bj K ij ( (cid:126)r ab ) = y ai (S4)We use convolutional filters up to L ≤ and rotation order for intermediate features of L ≤ .The radial functions are dense (fully-connected) neural networks acting on a finite radial basis. For example, a twolayer radial function would be expressed as R ( | r ab | ) = W kh σ ( W hq B q ( | r ab | )) (S5)0 Figure S1. Architecture of the Euclidean neural network. where B q are the set of radial basis functions; in this work, we use a finite set of Gaussian radial basis functions.The first two convolutional layers generate L = { , } atomic features and additional scalars to be used by followinggated blocks for nonlinearizing L = 1 pseudovectors [41]. The final convolution operation yields atomic features oforder L = 0 on each atom. Finally, states at all sites within the unit cell are aggregated into a one-dimensional array (cid:80) a ∈ N x ( q ) ia . We then apply a ReLU activation and normalize (by dividing by the maximum intensity) to predict thephonon density of states (DoS), which is simply an array of 51 scalars.1 DATA PRE-PROCESSING AND PREPARATION
The phonon DoS dataset with 1,521 crystalline solids calculated from density functional perturbation theory (DFPT)by Petretto et al. [40] is used for training and testing a predictive model in this study. We randomly split the entiredataset into training (80%), validation (10%), and test (10%) sets. We manually curated 3 additional experimentalphonon DoS from [54], adding the Cu and Ag examples to the training set and Au to the test set, in order toprovide the network examples of single-element compounds. The resulting training set had 1220 samples, and eachof validation and test sets had 152 samples.The DFPT-calculated phonon DoS data has high energy resolution, requiring a large number of parameters in theneural network to fit the output dimension. Given limited training data, it is challenging to train a predictive modelwith too many trainable weights. To ensure a balanced output dimension and resolution while retaining the mainfeatures of the phonon DoS, we interpolated the smoothed spectrum in the energy range ≤ ω ≤ − to 51points, corresponding to an energy resolution of 20 cm − . In particular, smoothing of phonon DoS is achieved byapplying a Savitzky-Golay filter of window length 101 and polynomial order 3, where filter parameters are determinedto best represent raw phonon DoS: getting rid of small fluctuations while retaining the main profiles. Representativeraw and smoothed phonon DoS curves are shown in Figure S2. Frequency (cm − ) P honon D o S ( a . u . ) NbInO , Frequency (cm − ) P honon D o S ( a . u . ) ScAlCO,
Frequency (cm − ) P honon D o S ( a . u . ) SiC,
Frequency (cm − ) P honon D o S ( a . u . ) Y Cl , Figure S2. Original and filtered phonon DoS.
HYPERPARAMETER OPTIMIZATION AND NEURAL NETWORK TRAINING
We randomly selected a set of initial hyperparameters within the ranges listed in Table S1 as a starting pointfor tuning. In addition to the listed parameters, we fixed the maximum radius r max to 5Å, which we found toadequately balance the number of edges created per vertex with the memory requirements to build the correspondinggraph. The selected value is also similar to the maximum bond length considered in the CGCNN [11]. We thensystematically tested hyperparameters within the specified ranges until an optimal set balancing validation loss andmemory limitations was found. The optimal set of hyperparameters, listed in the last column of Table S1, was thenused to train the final predictive model.The neural network is trained with a Quadro RTX 6000 GPU with 24 GB of RAM. We stop updating weights when2 Table S1. Hyperparameter searching range and optimization processHyperparameter Range Initial Selection Final SelectionMultiplicity of irreducible representation m a
16, 32, 48, 64 b
64 32Number of pointwise convolution layer 1, 2, 3, 4 3 2Number of basis for radial filters 5, 10, 15, 20 10 10Length of embedding feature vector 16, 32, 64, 128, 160 128 64AdamW optimizer learning rate c − , − , − , − − − × . k AdamW optimizer weight decay coefficient − , − , − , − − − a For outputs of first two convolutional layers only. b Out of memory is encountered at value 72. c We observe the lowest validation loss at a learning rate of − among all values tested, followed byoscillations, and thus adopt the exponentially decaying learning rate with k being the epoch number. Epochs M SE H i s t o r y TrainingValidation
Figure S3. The mean squared error versus epoch number. the validation loss stagnates and tends to rise to reduce over fitting; the loss history can be found in Figure S3.3
MORE PREDICTED PHONON DOS IN TEST SET
We present 100 predicted phonon DoS from the test set on opt of those appeared in the main text in Figure S4. Thesamples are sorted from top to bottom in order of increasing mean squared error (MSE). As discussed in the maintext, our model performs better in predicting energy ranges than exact amplitudes. While near-perfect predictionswere achieved for examples in the first few rows, the principal peaks and gaps of the phonon DoS were well-reproducedfor many materials located at bottom rows, such as MgSnAs , LiBeP, InGaO , and NaSbF . Frequency (cm − ) P honon D o S ( a . u . ) ZnS, MSE=0.0006 Ag HgI , MSE=0.0006 LaAgO , MSE=0.0007 YCuO , MSE=0.0010 ZnTe, MSE=0.0011 Ba TaBiO , MSE=0.0017 K Ga , MSE=0.0019 Cs Pd(IBr ) , MSE=0.0022 BaSr I , MSE=0.0024 K PtCl , MSE=0.0028 Cs As , MSE=0.0029 TlBr, MSE=0.0030 CsBr F, MSE=0.0033 Na CdP , MSE=0.0034 K PtBr , MSE=0.0035 Rb LiInCl , MSE=0.0035 BaTe , MSE=0.0035 Cs KBiF , MSE=0.0036 AlAs, MSE=0.0037 MgSe, MSE=0.0041 Rb BP , MSE=0.0041 Sr As O, MSE=0.0046 Sr(MgSb) , MSE=0.0049 AgBr, MSE=0.0049 CsCuO , MSE=0.0052 KLiTe, MSE=0.0065 K S , MSE=0.0068 Al CdTe , MSE=0.0070 Rb S, MSE=0.0071 RbSnCl , MSE=0.0073 CuBiSeO, MSE=0.0074 Rb Pd S , MSE=0.0079 KHfCuS , MSE=0.0079 NaSbS , MSE=0.0081 Sr GaGeN, MSE=0.0081 RbHgF , MSE=0.0082 BaAgTeF, MSE=0.0082 Cd(InTe ) , MSE=0.0084 Cs TiS , MSE=0.0086 AlN, MSE=0.0089 KI, MSE=0.0093 NaSrAs, MSE=0.0093 Cs RbYF , MSE=0.0097 Ca P O, MSE=0.0098 FeAs , MSE=0.0120 Cs O, MSE=0.0125 Fe(SiP) , MSE=0.0139 Ca PCl , MSE=0.0140 ScCoTe, MSE=0.0147 K SrTa O , MSE=0.0154 Cs CaH , MSE=0.0161 Rb O, MSE=0.0161 K Sn(AsS ) , MSE=0.0163 K NaInF , MSE=0.0165 Na ZnS , MSE=0.0179 Rb TlInF , MSE=0.0182 TiSiPd, MSE=0.0184 TiNiSn, MSE=0.0191 YAgO , MSE=0.0194 ZnPtF , MSE=0.0196 BaNaP, MSE=0.0209 CaMg(CO ) , MSE=0.0215 SrTiO , MSE=0.0217 RbNO , MSE=0.0244 Li As, MSE=0.0246 GaCuO , MSE=0.0263 GaAgO , MSE=0.0282 LiBr, MSE=0.0282 LiI, MSE=0.0294 BaLiP, MSE=0.0303 K Ti O , MSE=0.0314 AgP , MSE=0.0345 MgSnAs , MSE=0.0353 NaNO , MSE=0.0360 NbCu Se , MSE=0.0381 InS, MSE=0.0401 LaCoN , MSE=0.0429 LiBeP, MSE=0.0435 LiBeN, MSE=0.0469 Au, MSE=0.0517 CuRhO , MSE=0.0518 Y SeO , MSE=0.0541 TiGePt, MSE=0.0550 InGaO , MSE=0.0551 BaB F , MSE=0.0563 CuP , MSE=0.0573 Na SbS , MSE=0.0578 NaHS, MSE=0.0590 CuCl, MSE=0.0600 K Zr(BO ) , MSE=0.0708 LaN, MSE=0.0730 B, MSE=0.0784 PtO , MSE=0.0859 NaSbF , MSE=0.0950 BaMg FeH , MSE=0.0988 MgMoN , MSE=0.0988 NbBO , MSE=0.1155 NbNO, MSE=0.1167 Li ZrN , MSE=0.1251 LiH, MSE=0.1283
Figure S4. Predicted phonon DoS of randomly-selected examples from the test set, sorted from top to bottom in order ofincreasing MSE. ELEMENT REPRESENTATION
In Figure S5, we illustrate the number of occurrences of each element in the training, validation, and test sets.When compared to Figure 2b in the main text, it is found that elements corresponding to high MSE in Figure 2b ofthe main text tend to appear less frequently than those with lower errors, such as La, Ir, Mo, Rh, and Ru. A moredirect comparison is made in Figure S6. T i m eo f A ppea r an c e Ag Al As Au B Ba Be Bi Br C Ca Cd Cl Co Cr Cs Cu F Fe Ga Ge H Hf Hg I In Ir K La Li Mg Mn Mo N Na Nb Ni O Os P Pb Pd Pt Rb Re Rh Ru S Sb Sc Se Si Sn Sr Ta Tc Te Ti Tl V W Y Zn Zr Train Validation Test
Figure S5. Element representations in training, validation, and test sets.
Ag Al As Au B Ba Be Bi Br C Ca Cd Cl Co Cr Cs Cu F Fe Ga Ge H Hf Hg I In Ir K La Li Mg Mn T i m eo f A ppea r an c e TrainTest M ean S qua r ed E rr o r Mo N Na Nb Ni O Os P Pb Pd Pt Rb Re Rh Ru S Sb Sc Se Si Sn Sr Ta Tc Te Ti Tl V W Y Zn Zr T i m eo f A ppea r an c e M ean S qua r ed E rr o r Figure S6. Element representation in training set versus MSE in test set. The inset shows a scatter plot of the relationship. ADDITIONAL PHONON DOS PREDICTIONS OF MATERIALS WITH HIGHEST SPECIFIC HEATCAPACITY
We present E(3)NN-predicted materials (selected from 4,346 crystal structures without ground-truth DoS from theMaterials Project [49]) with the top 100 highest specific heat capacities in Figure S7.
Frequency (cm − ) P honon D o S ( a . u . ) Mg(AlH ) , C v =2544.19 Li HN, C v =2514.86 H NF, C v =2396.59 Li N, C v =2119.14 LiC, C v =2100.95 NaAlH , C v =1995.55 MgH , C v =1897.80 Li B C, C v =1897.56 HCN, C v =1871.45 HOF, C v =1681.57 NaHO, C v =1652.38 NaHO, C v =1648.66 Li Al , C v =1540.31 Li S N, C v =1508.12 CO , C v =1492.46 NaHCO , C v =1487.40 CaH , C v =1485.01 CuH (OF) , C v =1479.30 LiB(CN) , C v =1461.33 LiP(HO) , C v =1448.30 Mg(HO) , C v =1444.09 Li(CO) , C v =1412.19 CuH (OF) , C v =1409.28 Ca(HO) , C v =1364.96 LiBeBO , C v =1318.14 LiBO , C v =1310.19 Li Al , C v =1307.42 MgC , C v =1288.09 NaHF , C v =1266.08 NaGaH , C v =1258.21 Li H Rh, C v =1257.87 LiMgN, C v =1246.49 NaC, C v =1243.45 Li PClO , C v =1234.59 Li Ca, C v =1221.15 Li Ca, C v =1218.77 Be Mo, C v =1204.32 KHCO , C v =1156.81 Li FeN , C v =1132.57 AlHO , C v =1130.46 B O , C v =1129.18 Mg(BC) , C v =1121.04 KNa BN , C v =1099.76 CuB(CN) , C v =1094.76 Mg C , C v =1090.48 BN, C v =1088.57 NaMgBO , C v =1073.12 Cu(HO) , C v =1071.63 LiCo(CO) , C v =1070.94 KPH SO , C v =1069.43 Be, C v =1066.79 Li PBrO , C v =1060.68 Li FeN , C v =1056.63 MgB , C v =1047.31 Na N, C v =1043.93 KCN, C v =1043.74 NaBF , C v =1026.31 LiSO F, C v =1014.22 CaB , C v =1008.47 LiAlSi, C v =992.87 PH Br, C v =987.28 MgF , C v =976.68 AlBO , C v =973.06 Mg NF, C v =969.15 CaC , C v =968.47 CaC , C v =963.87 KLiO, C v =958.87 NaClO , C v =953.40 NaClO , C v =951.13 LiScF , C v =946.99 NaAlF , C v =946.64 CaC , C v =946.56 CsLiH SO , C v =925.82 CaN , C v =921.29 AlPO , C v =908.53 K CO , C v =904.47 CeH (CO ) , C v =899.39 NdH (CO ) , C v =895.72 Li TeO , C v =892.94 LiMnO , C v =890.33 SiO , C v =889.66 SmH (CO ) , C v =885.88 KO , C v =884.33 ScB , C v =879.61 B CCl O, C v =878.68 SiO , C v =872.07 KCd(NO ) , C v =865.97 Na CS , C v =861.74 KAlF , C v =855.20 K Mg Si O , C v =854.99 NaAlSi, C v =852.16 LiVO , C v =849.71 KClO , C v =845.14 NaMnF , C v =843.10 Na NiSO , C v =842.37 TiBO , C v =842.31 Na CuSO , C v =840.51 NaSi N , C v =840.30 Li (NiN) , C v =838.92 CsH O , C v =838.19 Figure S7. Top 100 high specific heat capacity materials, C V in unit J / (kg · K) . SPECIFIC HEAT CAPACITY CALCULATED FROM FULL ENERGY RANGES
Due to considerations about neural network size and spatial resolution of the predicted phonon DoS, our modelis trained with DFPT-calculated phonon DoS in the energy range [0 , − . When evaluating new materials,the E(3)NN-predicted phonon DoS is also confined within this domain. Therefore the specific heat capacities areevaluated from potentially incomplete phonon DoS.Here we present the specific heat capacities evaluated from the complete phonon DoS in Table S2. It is found thatmost materials have a slightly different C V , and 9 out of 12 points are still in good agreement. On the other hand,more dramatic discrepancies in the other 3 materials, MgH , H NF, and HCN, are partially induced by significantcontributions to the DoS at higher energy ranges outside those considered during training. The two-dimensionalhistogram and corresponding phonon DoS for the three materials are shown in Figure S8. A potential remedy for thisissue is to train our model with phonon DoS including higher energy ranges which may lose the energy resolution andis not necessary of majority of materials. The good agreement between E(3)NN-predictions and DFPT ground-truthspectra achieved in the energy range chosen for this work sufficiently demonstrates the potential for generalizabilityof our model.
Table S2. Comparisons of E(3)NN predicted and DFPT calculated specific heat capacities (in J / (kg · K) ).Material C E(3)NN V C DFPT V Material C E(3)NN V C DFPT V Li HN 2514.86 2320.06 Li S N 1508.12 1498.39H NF 2396.59 1578.65 LiBeBO Al Al Calculated C v P r ed i c t ed C v Frequency (cm − ) P honon D o S ( a . u . ) MgH H NFHCN
Figure S8. Left: Two-dimensional histogram comparing between specific heat capacities evaluated from E(3)NN-predicted andcomplete DFPT-calculated phonon DoS. Right: Full phonon DoS of three materials that display largest discrepancies.
COMPARISON TO CONVOLUTIONAL NEURAL NETWORK APPROACH
In order to better quantify the advantages of the tensor field neural network, we present a brief comparison to aconvolutional neural network (CNN). Here, the unit cells are encoded as three-dimensional densities, similar to [55].7
Table S3. Architectures of the convolutional neural network.Layer Input shape Output shape3D conv. layer, kernel size=5, strides=2, channel num.=1 (32,32,32,2) (16,16,16,1)activation=ReLU3D conv. layer, kernel size=3, strides=2, channel num.=2 (16,16,16,1) (8,8,8,2)activation=ReLU3D conv. layer, kernel size=3, strides=2, channel num.=4 (8,8,8,2) (4,4,4,4)activation=ReLU3D conv. layer, kernel size=3, strides=2, channel num.=8 (4,4,4,4) (2,2,2,8)activation=ReLUFlatten (2,2,2,8) 64Fully connected, activation=ReLU 64 64Fully connected, activation=ReLU 64 51
Epochs M SE H i s t o r y TrainingValidation
BaAgTeF BaTe K GeSe Cs TiS Cs O Rb LiInCl Sr(MgSb) LaAgO Y SeO TlBr GaAgO Cs CaH K PtCl Sr(ZnP) CuBiSeO ScCoTe Cs AlInH CdTe BaTeO TiSiPd Rb Pd S Li ZrN AlN NaCNO CaMg(CO ) NaSbF Li As NbBO Number of Sites ° ° ° M ean S qua r ed E rr o r True ¯ ! (cm ° ) P r ed i c t ed ¯ ! ( c m ° ) (a) (b) (c)(d) R e l a t i v e E rr o r Figure S9. Phonon DoS predicted by the convolutional neural network.
In particular, they are defined by ρ Z ( r ) = N (cid:88) i =1 Z i e −| r − r i | , ρ m ( r ) = N (cid:88) i =1 m i e −| r − r i | , (S6)and mapped onto a discrete three-dimensional grid with positions of grid points determined by corresponding latticevectors and parameters. The densities are then input into a three-dimensional convolutional neural network (archi-tecture is shown in Table S3). Same training, validation, and test samples are used here. In particular, to considerlattice periodicity, the densities for each unit cell are calculated in the middle of a3
In particular, they are defined by ρ Z ( r ) = N (cid:88) i =1 Z i e −| r − r i | , ρ m ( r ) = N (cid:88) i =1 m i e −| r − r i | , (S6)and mapped onto a discrete three-dimensional grid with positions of grid points determined by corresponding latticevectors and parameters. The densities are then input into a three-dimensional convolutional neural network (archi-tecture is shown in Table S3). Same training, validation, and test samples are used here. In particular, to considerlattice periodicity, the densities for each unit cell are calculated in the middle of a3 ×3
In particular, they are defined by ρ Z ( r ) = N (cid:88) i =1 Z i e −| r − r i | , ρ m ( r ) = N (cid:88) i =1 m i e −| r − r i | , (S6)and mapped onto a discrete three-dimensional grid with positions of grid points determined by corresponding latticevectors and parameters. The densities are then input into a three-dimensional convolutional neural network (archi-tecture is shown in Table S3). Same training, validation, and test samples are used here. In particular, to considerlattice periodicity, the densities for each unit cell are calculated in the middle of a3 ×3 ×3