Samuel S. Schoenholz

Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error

We investigate the impact of choosing regressors and molecular representations for the construction of fast machine learning (ML) models of 13 electronic ground-state properties of organic molecules. The performance of each regressor/representation/property combination is assessed using learning curves which report out-of-sample errors as a function of training set size with up to ∼118k distinct molecules. Molecular structures and properties at the hybrid density functional theory (DFT) level of theory come from the QM9 database [ Ramakrishnan et al. Sci. Data 2014 , 1 , 140022 ] and include enthalpies and free energies of atomization, HOMO/LUMO energies and gap, dipole moment, polarizability, zero point vibrational energy, heat capacity, and the highest fundamental vibrational frequency. Various molecular representations have been studied (Coulomb matrix, bag of bonds, BAML and ECFP4, molecular graphs (MG)), as well as newly developed distribution based variants including histograms of distances (HD), angles (HDA/MARAD), and dihedrals (HDAD). Regressors include linear models (Bayesian ridge regression (BR) and linear regression with elastic net regularization (EN)), random forest (RF), kernel ridge regression (KRR), and two types of neural networks, graph convolutions (GC) and gated graph networks (GG). Out-of sample errors are strongly dependent on the choice of representation and regressor and molecular property. Electronic properties are typically best accounted for by MG and GC, while energetic properties are better described by HDAD and KRR. The specific combinations with the lowest out-of-sample errors in the ∼118k training set size limit are (free) energies and enthalpies of atomization (HDAD/KRR), HOMO/LUMO eigenvalue and gap (MG/GC), dipole moment (MG/GC), static polarizability (MG/GG), zero point vibrational energy (HDAD/KRR), heat capacity at room temperature (HDAD/KRR), and highest fundamental vibrational frequency (BAML/RF). We present numerical evidence that ML model predictions deviate from DFT (B3LYP) less than DFT (B3LYP) deviates from experiment for all properties. Furthermore, out-of-sample prediction errors with respect to hybrid DFT reference are on par with, or close to, chemical accuracy. The results suggest that ML models could be more accurate than hybrid DFT if explicitly electron correlated quantum (or experimental) data were available.We investigate the impact of choosing regressors and molecular representations for the construction of fast machine learning (ML) models of thirteen electronic ground-state properties of organic molecules. The performance of each regressor/representation/property combination is assessed using learning curves which report out-of-sample errors as a function of training set size with up to

Structure-property relationships from universal signatures of plasticity in disordered solids

sim

Disconnecting structure and dynamics in glassy thin films

117k distinct molecules. Molecular structures and properties at hybrid density functional theory (DFT) level of theory used for training and testing come from the QM9 database [Ramakrishnan et al, {em Scientific Data} {bf 1} 140022 (2014)] and include dipole moment, polarizability, HOMO/LUMO energies and gap, electronic spatial extent, zero point vibrational energy, enthalpies and free energies of atomization, heat capacity and the highest fundamental vibrational frequency. Various representations from the literature have been studied (Coulomb matrix, bag of bonds, BAML and ECFP4, molecular graphs (MG)), as well as newly developed distribution based variants including histograms of distances (HD), and angles (HDA/MARAD), and dihedrals (HDAD). Regressors include linear models (Bayesian ridge regression (BR) and linear regression with elastic net regularization (EN)), random forest (RF), kernel ridge regression (KRR) and two types of neural net works, graph convolutions (GC) and gated graph networks (GG). We present numerical evidence that ML model predictions deviate from DFT less than DFT deviates from experiment for all properties. Furthermore, our out-of-sample prediction errors with respect to hybrid DFT reference are on par with, or close to, chemical accuracy. Our findings suggest that ML models could be more accurate than hybrid DFT if explicitly electron correlated quantum (or experimental) data was available.

Combining Machine Learning and Physics to Understand Glassy Systems

Behavioral universality across size scales Glassy materials are characterized by a lack of long-range order, whether at the atomic level or at much larger length scales. But to what extent is their commonality in the behavior retained at these different scales? Cubuk et al. used experiments and simulations to show universality across seven orders of magnitude in length. Particle rearrangements in such systems are mediated by defects that are on the order of a few particle diameters. These rearrangements correlate with the materials softness and yielding behavior. Science, this issue p. 1033 A range of particle-based and glassy systems show universal features of the onset of plasticity and a universal yield strain. When deformed beyond their elastic limits, crystalline solids flow plastically via particle rearrangements localized around structural defects. Disordered solids also flow, but without obvious structural defects. We link structure to plasticity in disordered solids via a microscopic structural quantity, “softness,” designed by machine learning to be maximally predictive of rearrangements. Experimental results and computations enabled us to measure the spatial correlations and strain response of softness, as well as two measures of plasticity: the size of rearrangements and the yield strain. All four quantities maintained remarkable commonality in their values for disordered packings of objects ranging from atoms to grains, spanning seven orders of magnitude in diameter and 13 orders of magnitude in elastic modulus. These commonalities link the spatial correlations and strain response of softness to rearrangement size and yield strain, respectively.

Machine learning determination of atomic dynamics at grain boundaries

Significance Do glassy dynamics depend strongly on local structure? In bulk systems, a quantitative answer to this question exists and is affirmative. The dynamical behavior of nanometrically thin glassy films is strikingly different from bulk systems, and it is natural to ask whether this difference stems from local structural differences. Using machine learning techniques, we show that altered dynamics near an interface do not stem from changes of local structure near the interface. Rather, the dynamics depend on the simultaneous occurrence of two independent processes, one that depends on structure but not position within the film, and an Arrhenius process that does not depend on structure but depends sensitively on position. Nanometrically thin glassy films depart strikingly from the behavior of their bulk counterparts. We investigate whether the dynamical differences between a bulk and thin film polymeric glass former can be understood by differences in local microscopic structure. Machine learning methods have shown that local structure can serve as the foundation for successful, predictive models of particle rearrangement dynamics in bulk systems. By contrast, in thin glassy films, we find that particles at the center of the film and those near the surface are structurally indistinguishable despite exhibiting very different dynamics. Next, we show that structure-independent processes, already present in bulk systems and demonstrably different from simple facilitated dynamics, are crucial for understanding glassy dynamics in thin films. Our analysis suggests a picture of glassy dynamics in which two dynamical processes coexist, with relative strengths that depend on the distance from an interface. One of these processes depends on local structure and is unchanged throughout most of the film, while the other is purely Arrhenius, does not depend on local structure, and is strongly enhanced near the free surface of a film.

Neural Message Passing for Quantum Chemistry

Our understanding of supercooled liquids and glasses has lagged significantly behind that of simple liquids and crystalline solids. This is in part due to the many possibly relevant degrees of freedom that are present due to the disorder inherent to these systems and in part to non-equilibrium effects which are difficult to treat in the standard context of statistical physics. Together these issues have resulted in a field whose theories are under-constrained by experiment and where fundamental questions are still unresolved. Mean field results have been successful in infinite dimensions but it is unclear to what extent they apply to realistic systems and assume uniform local structure. At odds with this are theories premised on the existence of structural defects. However, until recently it has been impossible to find structural signatures that are predictive of dynamics. Here we summarize and recast the results from several recent papers offering a data driven approach to building a phenomenological theory of disordered materials by combining machine learning with physical intuition.

Deep Information Propagation

Significance A machine learning method is used to analyze the atomic structures that rearrange within the grain boundaries of polycrystals. The method readily separates the atomic structures into those that rarely rearrange and those that often rearrange. The likelihood of an atom rearranging under a thermal fluctuation is correlated with free volume and potential energy but is not entirely attributable to those quantities. A machine-learned quantity allows estimation of the energy barrier to rearrangement for particular atoms. The grain boundary atoms that rearrange most have more possible rearrangement trajectories rather than much-reduced energy barriers, as in bulk glasses. The work suggests that polycrystal plasticity can be studied in part from the local atomic structural environments without traditional classification of microstructure. In polycrystalline materials, grain boundaries are sites of enhanced atomic motion, but the complexity of the atomic structures within a grain boundary network makes it difficult to link the structure and atomic dynamics. Here, we use a machine learning technique to establish a connection between local structure and dynamics of these materials. Following previous work on bulk glassy materials, we define a purely structural quantity (softness) that captures the propensity of an atom to rearrange. This approach correctly identifies crystalline regions, stacking faults, and twin boundaries as having low likelihood of atomic rearrangements while finding a large variability within high-energy grain boundaries. As has been found in glasses, the probability that atoms of a given softness will rearrange is nearly Arrhenius. This indicates a well-defined energy barrier as well as a well-defined prefactor for the Arrhenius form for atoms of a given softness. The decrease in the prefactor for low-softness atoms indicates that variations in entropy exhibit a dominant influence on the atomic dynamics in grain boundaries.