Active Learning A Neural Network Model For Gold Clusters \& Bulk From Sparse First Principles Training Data
Troy D Loeffler, Sukriti Manna, Tarak K Patra, Henry Chan, Badri Narayanan, Subramanian Sankaranarayanan
AActive Learning A Neural Network Model For GoldClusters & Bulk From Sparse First Principles TrainingData
Dr. Troy D Loeffler ∗ , Dr. Sukriti Manna ∗ , Dr. Tarak K Patra ,Dr. Henry Chan , Dr. Badri Narayanan , and Dr. Subramanian Sankaranarayanan † Center for Nanoscale Materials, Argonne National Laboratory , Lemont, Illinois 60439,United States Department of Mechanical and Industrial Engineering, University of Illinois , Chicago,Illinois 60607, United States Department of Chemical Engineering, Indian Institute of Technology Madras , Chennai,TN 600036, India Department of Mechanical Engineering, University of Louisville, Louisville , KY 40292,USAJune 9, 2020
Abstract
Small metal clusters are of fundamental scientific interest and of tremendous significance in catalysis.These nanoscale clusters display diverse geometries and structural motifs depending on the cluster size; aknowledge of this size-dependent structural motifs and their dynamical evolution has been of longstandinginterest. Given the high computational cost of first-principles calculations, molecular modeling andatomistic simulations such as molecular dynamics (MD) has proven to be an important complementarytool to aid this understanding. Classical MD typically employ predefined functional forms which limitstheir ability to capture such complex size-dependent structural and dynamical transformation. NeuralNetwork (NN) based potentials represent flexible alternatives and in principle, well-trained NN potentialscan provide high level of flexibility, transferability and accuracy on-par with the reference model used fortraining. A major challenge, however, is that NN models are interpolative and requires large quantities( ∼ or greater) of training data to ensure that the model adequately samples the energy landscape bothnear and far-from-equilibrium. A highly desirable goal is minimize the number of training data, especiallyif the underlying reference model is first-principles based and hence expensive. Here, we introduce anactive learning (AL) scheme that trains a NN model on-the-fly with minimal amount of first-principlesbased training data. Our AL workflow is initiated with a sparse training dataset ( ∼ ∼
500 total reference calculations. Using an extensive DFT test set of ∼ ∗ These two authors contributed equally † Corresponding author: [email protected] a r X i v : . [ phy s i c s . c o m p - ph ] J un Introduction
Small clusters approaching the sub-nanometer size range have attracted a lot of interest in catalytic appli-cations. [1, 2] Such size-selected clusters that comprise of a handful of atoms often display exotic catalyticproperties that are much different than that of either nano-sized or bulk catalysts. [3, 4, 5, 6, 7, 8, 9] Theseclusters contain well-defined number of atoms and offer an ideal platform to study catalysis at the atomiclevel. They also serve as model systems to enable a comprehensive fundamental insight into the nature of thecatalytic processes that are otherwise difficult to explore using catalysts prepared by conventional methodsthat often yield particles with finite distributions in size and composition.The recent advances in synthesis science has allowed us to exercise precise control over the structureand composition of these small catalytic clusters. For example, Vajda and coworkers have shown thatsub-nanometer Pt clusters can serve as highly active and highly selective catalyst for the oxidative dehy-drogenation of propane. [10] More recently, they have also shown that sub-nanometer sized cobalt oxideclusters can enable oxidative dehydrogenation of cyclohexane at lower temperatures than conventional cat-alysts, while eliminating the combustion channel. [11] These individual clusters that contain a handful ofatoms have a high surface-to-volume ratio and much higher fraction of undercoordinated atoms. Apart fromdisplaying exceptional catalytic activity, they also offer an excellent and economic utilization of the metalloading. [12] In view of these studies, an area of growing interest is to design new catalytic materials inan atom-by-atom fashion. A lot of catalyst design work focuses on exploring conditions and pathways fortheir synthesis and are effectively aimed at tuning the number of under-coordinated sites via experimentalcontrols such as pressure, temperature etc. From this perspective, physically accurate, flexible and accurateMD simulations and models are important to enable insilico design given the exhaustive space that needs tobe explored and the experimental trials being time-consuming and costly. The recent advances in compu-tational resources and first-principles based methods have allowed for rapid high-throughput computationalstudies to design catalysts. [13, 14, 15] More recently, the advances in data science and machine learning haveallowed for computations to provide a better characterization to complement experiments and extract moreinformation about the structure and compositions of these catalyst. [16] Computations based on densityfunctional theory have allowed us to uniquely explore the energetics and thermodynamics of high-energyintermediates or transient metastable states that play an important role in the catalytic pathway that mayescape experimental characterization. [17, 18, 19]Apart from energetics, the dynamical evolution of these clusters is also important from a design per-spective. These clusters undergo dynamical processes that involve structural transitions from one sta-ble/metastable state to another; often these metastable states have been shown to display much highercatalytic activity than their stable counterparts. [20] Ab-initio molecular dynamics (AIMD) techniques rep-resents a popular method to probe the dynamics. But despite the improvements in computational resources,the AIMD simulations are limited in the timescales and length-scales that they can access. Furthermore,it is also worth noting that the the global minimum energy configurations of these catalytic clusters in themid-size regime ( n = 20-100) are not well understood. Such exhaustive structural searches for these sizes re-main intractable within the framework of high-fidelity calculations such as DFT even with the most efficientsampling methods (e.g., evolutionary algorithms, [21, 22] basin-hopping, [23, 24, 25] etc.).A classical description of the potential energy surface of these small clusters can provide a cheapersurrogate to perform either longer time dynamical simulations or carry out an exhaustive search of thestructure/compositional space of these catalytic clusters. The primary challenge with these models is thatthey trade accuracy for computational efficiency. Despite being popular, classical models with pre-definedfunctional forms struggle to accurately describe the structure and dynamics of clusters in the ( n = 10-100)range. For instance, Au clusters in the sub-nanometer range undergo a planar-to-globular transition atcluster size of 13 atoms, which has proven to be very difficult for empirical potential models to capture. [26]Spherically symmetric potentials such as embedded atom method and Sutton Chen potentials cannot capturethe planar configurations whereas bond-order potentials such as Tersoff perform well for planar structuresbut do not completely capture the size dependent structural transition in Au clusters. [27] It is well knownthat the use of predefined functional form imposes serious limitations on the physics and chemistry that canbe captured.Neural network (NN) based potential models offer a flexible alternative to capture the size dependentstructural and dynamical transformations in these nano and sub-nanoscale catalysts. [28, 29, 24, 30] Recently,2N models are emerging as a popular technique due to the rapid advancement in the computational resourcesas well as the myriad of electronic structure codes that allow for efficient generation of the training data. [31]The underlying goal in the development of these NN models is to train against vast amounts of high-fidelityfirst-principles data and thereby replicate their accuracy at a fraction of their computational cost. Aninherent limitation of these NN models is that they are interpolative and as such the traditional approachfor training a NN has often relied on generating as large a training data as is possible. Such large-scalegeneration of high-fidelity training data can become challenging depending on the level of the electronicstructure calculations employed. [32]To address the issue with training data generation, there have been several recent efforts to deviceactive learning strategies that allow for efficient sampling of training data for NN models. Smith et al.employed an active learning (AL) strategy based on the Query by Committee (QBC) scheme.[33] QBC usesthe disagreement between an ensemble of ML potentials to infer the reliability of the ensemble’s prediction.QBC allowed for automatic sampling of the regions of chemical space where the potential energy was notaccurately described by the ML potential. Their AL approach was validated on a test set consisting of adiverse set of organic molecules and their results showed that one requires only 10% to 25% of the data toaccurately represent the chemical space of these molecules.Similarly, Zhang et al. [34] introduced an AL scheme (deep potential generator (DP-GEN)) that constructsML models for simulating materials at the molecular scale. Their procedure involve exploration, generationof accurate reference data, and training. They used Al and Al-Mg as representative cases and showed thatML models can be trained with minimum number of reference data. In another work, Vandermause et al. [35]sampled structures on-the-fly from AIMD and used an adaptive Bayesian inference method to automate thetraining of low-dimensional multiple element interatomic force fields. Their AL framework uses internaluncertainty of a Gaussian process regression model to decide acceptance of model prediction or the need toaugment training data. In all of the above studies, the overarching aim in these studies is to minimize theab-initio training data required to describe the potential energy surface.Here, we introduce a new active learning (AL) strategy [36, 37, 38] that learns the potential energysurface description from minimal amount of first-principles training data sampled from on-the-fly MonteCarlo simulations. Our workflow starts with minimal training data ( ∼ to be globular [44] in contrast to previous DFT calculations that show Au to be pla-nar. [45, 46, 47] Given these challenges, gold catalytic clusters represent an excellent system for testing theefficacy of our AL scheme. We show that our AL-NN is able to adequately represent the energy landscapefor diverse sizes and geometries as well as the dynamical properties of both clusters and bulk by samplingminimal amount of reference data ( ∼
500 total reference data).
Our AL strategy is shown schematically in Fig. 1 and involves the following major steps: (1) Training of theNN using the current structure pool (of Au nanoclusters configurations). (2) Running a series of stochasticalgorithms to test the trained network’s current predictions. (3) An identification of configurational spacewhere the NN is currently struggling. (4) An update of the structure pool with failed configurations. (5)Retraining of the NN with the updated pool and back to step 2. To test our AL scheme, we train a neural3etwork to a reference DFT-PBE energetics for several gold configurations. The neural networks used in thisstudy were constructed and trained using the Atomic Energy Network (AENet) software package,[48] whichwas modified to implement the active learning scheme outlined above. Simulations using these networkswere carried out using AENet interfaces with the Classy Monte Carlo simulation software[49] to perform theAL iterations. The main steps in our active learning iteration include:
Convergence check (NN vs. reference)
Identify failed configurationsUpdate structure poolTrain NN (structures vs. energies)
Test NN predictions (stochastic algorithms)
Sample 5 structures Optimized NN potential
Figure 1: Schematic showing the active learning workflow employed for generation of the NN potential modelfor Au nanoscale catalysts.
The Vienna Ab-initio Software Package (VASP) [50] with the Perdew-Burke-Eznerhof (PBE) [51] exchange-correlation functional was used to perform all the density functional theory calculations. The spin polariza-tion was included in this DFT calculations. For element gold projector-augmented wave (PAW) potentials(PAW PBE Au 04Oct2007) provided with VASP were used. A single k -point at the center of the Brillouinzone was used for each calculation. Gaussian smearing with a width of 0.001 eV was used to set partialoccupancies. The convergence criteria for the electronic self-consistent iteration and the ionic relaxation loopwere set to be 0.1 meV and 1 meV per cluster, respectively.To evaluate the equation of state (EOS) plot for gold fcc gold lattice, ±
5% strain was applied inall three directions. The initial bulk structure of gold fcc system has been collected from materials projectdatabase. [52] A dense k -point grid, defined by n atoms × n kpoints ≈ n Atoms is the number of atomsin the primitive cell and n atoms × n kpoints is the number of k -points were used in the DFT calculations forEOS plot. A relatively high tolerance of 10 − eV for energy convergence was employed in these calculations.Three independent elastic constants, n atoms × n Kpoints ≈ ε , (defined by equation1) in such a way that the new lattice vectors r in the distorted lattice is given by r (cid:48) = ( I + ε ) r where I isthe unit matrix. ε = e e e e e e e e e (1) Our NN consists of four layers of neurons; all the neurons/nodes of a layer are connected to every node inthe next layer by weights in the manner of an acyclic graph. The two intermediate layers (hidden layers)4able 1: Three strain combinations in the strain tensor for calculating the three elastic constants ( C , C ,and C ) of cubic structure of fcc gold. The magnitude of applied strain is varied in steps of 0.005 from δ =-0.02 to 0.02. ∆ E is the difference in energy between that of the strained lattice and the unstrainedlattice. The unstrained lattice volume is V .Strain Parameters (unlisted e i = 0) ∆ E/V e = e = δ, e = δ ) − C − C ) δ e = e = e = δ ( C C ) δ e = δ, e = δ (4 − δ ) − C δ consist of 10 nodes each. The input layer has 26 nodes which hold 26 symmetry functions that represent co-ordinates of the gold’s potential energy surface (PES). The output layer consists of one node that representsthe potential energy of a gold atom in a given configuration. Besides, the input layer and the hidden layerscontain a bias node that provides a constant signal to all the nodes of its next layer. The choice of thisnetwork topology is based on a large number of trials for capturing various thermophysical properties ofgold clusters. The Cartesian coordinates of a given gold atom are mapped into rotational and translationalinvariant co-ordinates as G i = (cid:88) j e − η ( r ij − R s ) · f c ( r ij ) (2) G i = 2 − ζ N (cid:88) j,k (cid:54) = i (1 + λ cos θ ijk ) ζ · e − η ( r ij + r ik ) · f c ( r ij ) · f c ( r ik ) (3)Here, f c ( r ij ) = 0 . πr ij R c ) + 1] for r ij < R c and f c ( r ij ) = 0 . r ij corresponds to the distance between i th and j th particles of a gold cluster and θ ijk is the angle formed by r ij and r ik . The indices i , j and k run over all the particles in a cluster, which are with within a cut-offdistance R c = 6 . A . We have used 8 radial symmetry functions G each with a distinct value of η , which aretabulated in Table 2. Similarly, 18 angular symmetry function are used, each with a distinct set of values.The parameters of these 18 angular symmetry functions are reported in Table 2. The functional forms ofthese symmetry functions (Behler-Parrinello type symmetry functions [60]) have been used successfully toconstruct PES of different molecular systems, and thus adopted for this work.Table 2: Parameters of the 8 radial symmetry functions G and 18 angular symmetry functions G with acut-off distance of 6.0 ˚AIn this work, each and every atoms of a gold cluster is represented by a NN and the total energy of thecluster is defined as E = (cid:80) N A i E i , where E i is the output of the i th NN, and NA is the total number of gold5toms in a given cluster which is same as the number of NNs. We note that the architecture and weightparameters of all these atomic NNs are identical. During the training, the symmetry functions of each atomof a configuration are fed to the corresponding NN via its input layer. In every NN, all the compute nodesin the hidden layers receive the weighted signals from all the nodes of its previous layer and feeds themforward to all the nodes of the next layer via an activation function as x ij = f ( (cid:80) k W ik,j x ( i − ,k ). Here, f ( x ) = tanh( x ) is used as the activation function of all the compute nodes. As mentioned earlier, the sumof all the outputs from all the NNs serves as the predicted energy of the system. The error in the NNs,which is the difference between the predicted and reference energies of a given configuration, is propagatedbackward via the standard back-propagation algorithm. All the weights that connect any two nodes areoptimized using the Levenberg-Marquardt method[61] in order to minimize the error, as implemented withinthe framework of AEnet[48] open-source code. N eu r a l N e t w o r k E ne r g y [ e V / a t o m ] − − − − − − − − − − (a) (b) M ean A b s o l u t e P r e d i c t i on E rr o r [ m e V / a t o m ] N u m b e r o f T r a i n i n g S t r uc t u r e s Active Learning Iteration0 10 20 30 40 5010 Figure 2: Active learning of a NN potential for gold nanoclusters from sparse first-principles data. (a) Themean absolute error of the AL-NN tested on the DFT test set is plotted as a function of active learningiteration or generation (solid red dots). The scale on the RHS of the plot shows the size of the training data(solid blue dots) for the same training generation. (b) A correlation plot showing the performance of thefinal optimized network on the 579 structure training set.
A Levenberg-Marquardt approach[62] was used to optimize the neural network weights for each AL gener-ation. This was done with a batch size of 32 structures and a learn rate of 0.1 once the structure pool waslarge enough to accommodate these settings. Initially, the batch size was set to 1, given the small initialtraining data set. For each network generation, the neural network is trained for a total of 2,000 epochs,where each epoch represents one complete training cycle. The AENet makes use of a k -fold cross validationscheme, where a given fraction ( k ) of the training set is not used for the objective minimization. Insteadthis fraction is used to cross validate the training process to minimize over-fitting. For each AL iteration,the network which had the best error from the cross validation was chosen as the best network for this ALiteration and is carried forward. Once the best network has been chosen, a series of simulations are run to actively sample the configurationalspace predicted by the current NN. It was found that MD is not suitable for sampling within this schemedue to the fact that when the network is still in its infancy, large spikes in the forces can lead to unphysicalacceleration of particles within the simulation box. In addition, even in a reasonably well-trained network,MD can be trapped in a local energy well that prevents it from searching the phase space outside of this well.This can often create models that work well within the trained local minima, but can have catastrophically6ad predictions when the model is applied to environments found outside of the training set. Monte Carloand other similar sampling methods in contrast are much less sensitive to spikes in the energy surface whichmake them more suitable methods for sampling poorly trained energy landscapes.In addition, a wide collection of non-physical moves or non-thermal sampling approaches can be used. Forthe purposes of this work, Boltzmann based Metropolis sampling and a nested ensemble based approach [63]were used to generate the structures for each AL iteration. This was done to gather information on boththermally relevant structures predicted by the neural network as well as higher energy structures which maystill be important for creating an accurate model. The Metropolis simulation was run for 5,000 MC cycles at300K with the initial structure being randomly picked from the current neural network training pool. TheNested Ensemble simulations were run for another 5,000 cycles.Figure 3: Performance of the actively learned NN model on an extensively sampled test data set. Energycorrelations comparing actively learnt NN-prediction with the reference DFT energies for a test set thatcomprises of ∼ are provided in the inset of the plot. After the stochastic sampling step is completed, a set of 10 structures are gathered from the trajectoryfiles of the Metropolis and Nested Sampling files. These are sampled by outputting a structure every 1000number of cycles for both the nested ensemble run and metropolis. For the nested sampling run this is setup such that we pull one structure from each energy “strata” as the nested sampling gradually constricts theenergy space. This ensures we are always testing structures from both high energy and low energy regionsof the phase space. The real energy of these structures are computed using DFT-PBE and compared withthe predictions of the NN model. For each structure, if the neural network and the DFT prediction do notagree within a given tolerance, the structure is then added to the training pool to be used for the next ALiteration. This entire process is continued until the exit criteria is hit. For this work, we specified that ifno new structures were added in 5 consecutive AL iterations, that the potential has converged. For theaddition tolerance, we specified that any structure with a greater than difference of 20 meV between the realand predicted energy should be added to the training pool. The acceptable tolerance is based on typicalprediction errors of DFT (the reference model) which is around 20 meV. [64] Also, the kT value for roomtemperature is ∼
25 meV - so the errors are within typical thermal fluctuations at room temperature.7 lanar Globular Icosahedra
G1G2G3G4G5P2P1 G1G2G3G4G5P2P1 C o h e s i v e E n e r g y [ e V / a t o m ] − − − − − − − Atomic Structures Ih DFTNeural Network
Figure 4: Predictive power of the AL-NN for the various 2D and 3D Au configurations with respect to theDFT-predicted global energy minimum structure. The cohesive energies of planar structures, intermediateconfigurations and 3D icosahedron (Ih) computed with AL-NN are compared with those obtained by DFT.The blue and red solid lines correspond to the DFT predicted and Neural Network predicted cohesive energiesrespectively.Most of the available EFFs predict the globular Ih to be the most stable structure for Au in contrast toDFT (which predicts planar to be the global energy minimum). AL-NN describes the energetics of Au clusters in excellent agreement with DFT calculations. The initial neural network cannot be trained on zero data, a single structure is used to seed the initial neuralnetwork in order to kick off the training process. This was chosen to be a reasonably minimized structurein order to ensure at least one low energy configuration was contained in the training set. Theoreticallyone could begin with any number of seed structures, but for the purposes of evaluating the efficiency ofthis approach, the absolute minimal seed data was used. In order to rigorously validate the neural networkmodels, we created a test set that consists of roughly ∼
500 configurations of Au clusters.
First, we evaluate the performance of our active learning (AL) scheme depicted in Fig. 1. Fig. 3 showsthe mean absolute error (MAE) in meV/atom as a function of epochs – each epoch is an AL iteration or acomplete training cycle. Fig. 3(a) also shows the number of structures added during each of the AL iteration.Our training of the gold NN is initiated with minimalist number of training configurations. Therefore, theinitial NN have very high errors ∼ ∼ ∼
20 meV/atom at ∼
50 epoch. Initial trainingerrors are nearly on the same magnitude as the total system energy. The NN learns rapidly in the beginningas more distinct (failed) cluster configurations are added to the pool. The MAE drops sharply and plateausat AL iteration ∼
10 suggestive that the NN search is stuck at a local minimum. After about a total of 50AL epochs or iterations, we see that the MAE drops to ∼
20 meV/atom. At this point, the gold NN reachesour prescribed stopping criteria i.e. no new structures are added during 5 consecutive test cycles. It is worthnoting that final structure count at this point has reached a total of ∼
500 unique training structures. Fig. 2plots the correlation between the performance of the final optimized NN trained on the ∼
579 configurationtraining set. The AL-NN predictions of energetics for the clusters in the training pool are compared withthat the reference DFT model. MAE for the training set was found to be less than 20 meV/atom, which is8igure 5: The size-dependent transition of a gold cluster at 300 K as predicted by the NN developed in thisstudy. The atoms are introduced one-by-one into the system via grand canonical swap moves. The MonteCarlo simulation was run for a total of 100,000 cycles at 300K and a constant gas phase reservoir density.of the same order of magnitude as DFT error.We next evaluate the network performance as a function of the number of AL epochs. We choose the bestnetwork from each AL iteration and test its performance on a test set that comprises of 1101 configurationsand their energies. Fig. 2 (a) shows the correlation between AL-NN predicted energies vs. reference DFT-PBE energies. As expected, we find that the final optimized NN is able to reliably predict the Au clusterenergies for an elaborate test data set generated not only near equilibrium, but also in the highly non-equilibrium region that extends far beyond. As a more rigorous test of the performance of AL-NN, wecompute the forces on the atoms for the various clusters and compare those with that obtained from DFT-PBE. It should be noted that the forces were not included as part of the training during the AL iterations.Fig. 2 (b) shows the correlation between the AL-NN predicted vs. DFT forces. Each point in this correlationplot represents one of the force components - F x , F y and F z acting on a particle. We find the overall MAEbetween AL-NL vs. DFT predicted forces is ∼
20 meV/˚A. Given that the NN had not been trained onthe forces, this agreement with the DFT is of excellent quality. Overall, our Au AL-NN optimized networkperforms very well over an extensively sampled test data set.We also test the performance of AL-NN in reproducing the DFT predicted energetic ordering of structuralisomers at a given cluster size. Fig. 4 compares the energetic ordering of representative planar, intermediateand globular isomers of Au clusters predicted by AL-NN with those from DFT calculations (atomic co-ordinates were relaxed at the corresponding level of theory). The Au isomers depicted are configurationsthat lie near the global energy-minimum as predicted by DFT. The AL-NN predict the correct energeticordering consistent with the DFT predictions although it should be noted that the energetic difference be-tween various isomers are slightly underpredicted. This performance is still much better than those of mostpopular existing empirical force-fields such as EAM and its variants which incorrectly predict the icosahedralstructure to be more stable than planar for Au .A crucial yet challenging test of any FF is its ability to accurately capture the size dependent global mini-mum energy (GM) configuration especially for cluster sizes that are not part of the training set. Such as testcan be regarded as a true test of its transferability. The AL-NN is successful in predicting GM configurationsfor nanoclusters at several sizes as shown in Fig. 5. In accordance with the DFT predictions, our AL-NNGM structures are planar for clusters with size n <
14 atoms and globular at higher sizes. Our predictedcritical size for planar to globular transition (13 atoms) is identical to previous DFT calculations, [26, 27]and previously reported ion mobility and spectroscopy measurements. [67] AL-NN predicted planar GMstructures for sizes up to 13 atoms match the ones reported in Ref. [68, 69] using DFT calculations. AL-NNalso successfully reproduce the evolution of various structural motifs with cluster size in accordance withDFT predictions and spectroscopic experimental observations. [68, 69, 67]9able 3: Structural, energetic, and elastic properties of bulk polymorphs of gold as predicted by the AL-NN model developed in this study. These predictions are compared with values obtained from our DFTcalculations, and previous experiments (if available). E fccC refers to cohesive energy of FCC, a j to latticeparameter of cubic polymorph j , and ∆ E j − fcc is the difference of cohesive energy between polymorph j andFCC. The quantities C ij are the values of elastic stiffness constants. Neural Network EAM [39] SC[65] ReaxFF[44] HyBOP[27] DFT Experiment(This Study) (This study) E fccC (eV/atom) -3.22 -3.93 -3.78 -3.77 -3.82 -3.22 -3.81[39] a fcc (˚A) 4.15 4.08 4.08 4.18 4.19 4.17 4.07[39]∆ E bcc − fcc (eV/atom) 0.02 0.02 0.03 0.14 0.08 0.02 - a bcc (˚A) 3.306 3.24 3.25 3.31 3.32 3.31 -∆ E sc − fcc (eV/atom) 0.32 0.39 0.28 0.69 0.5 0.20 - a sc (˚A) 3.08 2.65 2.72 2.95 2.82 2.76 -∆ E dia − fcc (eV/atom) 1.22 0.94 0.60 1.0 1.37 0.71 - a dia (˚A) 6.83 5.75 6.07 6.72 6.45 6.18 - C (GPa) 171 183 180 168 231 150 192[66] C (GPa) 157 159 148 130 170 129 163[66] C (GPa) 42 45 42 55 75 31 42[66] While the snapshot energies show that the examined planar structures are lower in energy than theglobular counter parts, an actual molecular simulation is required to check for other unknown states thatmight potentially be lower. To examine this a Grand Canonical Ensemble Monte Carlo simulation wasperformed using the Aggregation Volume Bias Monte Carlo approach. In these simulations the cluster isslowly grown atom by atom along with standard Monte Carlo moves in order to observe how the clusterconfigures itself under thermal motion. The snapshot results of these simulations can be found in Fig. 5. Ingood agreement with the snapshot data, the cluster stably grows in a planar configuration up to 13 atoms insize. As the cluster growth further the addition of atoms onto the top of the 2D structure can be observed.Overtime more and more atoms are added in out of plane positions and the cluster begins to fold in on itselfand begins to form a cage structure. Our simulations predicts the correct trend in the geometry as a functionof cluster size. Because of the statistical improbability of forming 2D clusters using 3D Monte Carlo movesand given that 3D is more entropically favored than 2D, we can safely conclude that planar structures arestructurally stable under thermal conditions.Apart from describing the clusters accurately, it is also essential for the AL-NN to appropriately describethe bulk as many fundamental research problems including adsorption on Au surfaces, diffusion of clusterson surfaces of bulk Au, and breakdown of large Au clusters into small ones upon high energy impact. Fig.6 shows the comparisons of the equation for state for bulk Au compared to that obtained from DFT. Wenote that the agreement between the AL-NN predictions and DFT is excellent. The structural, elastic,and cohesive energies of various bulk Au polymorphs predicted AL-NN with those from DFT/experimentsare summarized in Table 3. AL-NN preserves the DFT evaluated energetic ordering of the various bulkpolymorphs and predicts the FCC to be the most stable bulk polymorph in agreement with previous DFTand experiments. AL-NN also predicts lattice parameter of FCC Au to be 4.15 ˚A, in good agreement withour DFT calculations (4.17 ˚A), and previous experiments (4.07 ˚A).[39] The cohesive energy for FCC Aupredicted by AL-NN (-3.22 eV/atom) agrees very well with previous experiments (-3.81 eV/atom).[39] Notethat DFT-PBE significantly underestimates this value (-2.97 eV) due to its inadequate treatment of thedispersion effects in Au.[46, 26] AL-NN predictions for the elastic constants are in excellent agreement withexperiments as well as spherically symmetric potentials (i.e., EAM, SC). In particular, one of the challengingelastic properties to describe is the ratio C /C . Our AL-NN predicts this ratio to be 3.78 in excellentagreement with experimental value of 3.9 and DFT value of 4.1. EAM and SC perform well for this ratiogiving value ∼ FTNeural NetworkMAE: 3.07 meVRMSE: 3.62 meVR : 0.996 Δ E ( m e V ) − − − − − Figure 6: Comparison of the AL-NN equation of state with that obtained from DFT. The calculated MAE,RMSE and R are provided in the inset of the plot. The AL-NN training approach has shown it is capable of creating a NN that does an excellent job ofreplicating DFT energies using relatively small training data sets. Only performing on the order of a fewhundred DFT calculations, we build an NN model that was not only able to capture the cluster propertiesbut also perform well on bulk test set (despite the bulk properties not being included in the training data).We next test the effectiveness of the nested ensemble sampling scheme vs. a random sampling (seeFig. 7). For comparison, we generated a neural network whose training set was created by a mixture ofrandom structure generation and thermal sampling using a MD force-field. An identical number of structureswere randomly sampled similar to that during iterative AL runs and used to train a neural network. TheRandomly Generated Structure (RGS) approach created a network that had a MAE of 164.3 meV/atomerror with many of the energies grossly over predicted compared to the reference DFT model. On the otherhand, AL approach has an MAE of 26.4 meV/atom for the same number of training structures. This clearlyshows that the nested ensemble search performs much better compared to a random search.We next assess the reproducibility as well as the influence of the starting configurations on the NNevolution during the AL cycle. To show the reproducibility of the AL algorithm, we perform an ensembleof 30 AL runs. The results are plotted in Fig. 8 which shows the average MAE error against the validationset as a function of AL iteration (blue curve). The error bounds are given as the worst case (lower) and thebest case (upper) scenario of the 30 independent AL runs. All 30 runs converge after ∼
35 AL iterations -the worst network has ∼
35 meV/atom error and the best network ∼
20 meV/atom. This clearly showsthat regardless of the starting configuration, the AL scheme converges to an optimal network after samplinga few hundred Au cluster configurations.Finally, there are at least two main computational cost advantages to our AL procedure. First, the totalcost for training the AL-NN with a training data size increasing up to 500 Au cluster configuration is approx.30-40 core-hrs. The same for training a conventional NN with ∼ ∼ ∼
240 core-hrs. Given that the training data size for a conventional NNis ∼ ∼ adomly Generated Training Set (RGS)Active Learning (NN)MAE (RGS) : 164.3 meV/atomMAE (NN) : 26.4 meV/atom N eu r a l N e t w o r k E ne r g y [ e V / a t o m ] − − − − − − − − − − − − Figure 7: Energy correlation plot for randomly generated training set vs. AL generated training set. Themean absolute error (MAE) are provided in the inset.quantum calculation as well as the cluster sizes being sampled. The computational cost savings are clearlymuch more for higher fidelity calculations such as CCSD and QMC and for larger cluster sizes in the trainingdata.
In summary, we introduce an automated active learning workflow for building NN models against a sparsefirst-principles training dataset constructed on-the-fly. Our AL scheme allows for on-the-fly sampling ofboth the configurational and potential energy surface of gold nanoclusters of different sizes and builds ahigh-quality neural network with a sparse dataset that comprises of only ∼
500 reference DFT evaluations.Using an extensive DFT test set of 1101 configurations, we show that our NN provides excellent predictionsof both the energies and forces over a wide variety of cluster sizes (that were not originally part of thetraining set). Our AL trained NN captures the global minimum energy configurations from several differentsamples of cluster sizes and the energetic ordering i.e. stability of various cluster configurations at any size.It also captures the size dependent critical size of transition from planar to globular clusters consistent withDFT calculations. The NN also predicts the evolution of structural motifs with cluster size. Moreover, italso reasonably captures the thermodynamics, structure, elastic properties, and energetic ordering of bulkcondensed phases, in excellent agreement with DFT calculations and previously reported spectroscopic ex-periments. Given that high-fidelity quantum calculations such as quantum Monte Carlo (QMC)[70] andcoupled clusters (CCSD)[71] are computationally expensive and hence, even with the significant improve-ments in computing resources, one can only generate sparse data sets. From this perspective, our AL schemeovercomes a major limitation of training NN against sparse datasets. Finally, our work lays the groundworkfor future construction of NN to describe the complex potential energy landscape and dynamics of catalyticnanoclusters by training against sparse high-fidelity data obtained from minimal quantum calculations.
Acknowledgement
We acknowledge funding from BES Award de-sc0020201 by DOE to support thisresearch. The use of the Center for Nanoscale Materials, an Office of Science user facility, was supportedby the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Contract No.12 ean A b s o l u t e P r e d i c t i on E rr o r [ m e V / a t o m ] Active Learning Iteration5 10 15 20 25 30 35 40 45 50
Figure 8: Average MAE error against the validation set as a function of AL iteration (blue curve) and theerror bounds (red region).DE-AC02- 06CH11357. This research used resources of the National Energy Research Scientific ComputingCenter, which was supported by the Office of Science of the U.S. Department of Energy under Contract No.DE-AC02-05CH11231. An award of computer time was provided by the Innovative and Novel ComputationalImpact on Theory and Experiment (INCITE) program of the Argonne Leadership Computing Facility atthe Argonne National Laboratory, which was supported by the Office of Science of the U.S. Department ofEnergy under Contract No. DE-AC02-06CH11357. SKRS acknowledges UIC start-up funds for supportingthis research.
References [1] Lichen Liu and Avelino Corma. Metal catalysts for heterogeneous catalysis: from single atoms tonanoclusters and nanoparticles.
Chem. Rev. , 118(10):4981–5079, 2018.[2] JF Hamilton and RC Baetzold. Catalysis by small metal clusters.
Science , 205(4412):1213–1220, 1979.[3] Eric C Tyo and Stefan Vajda. Catalysis by clusters with precise numbers of atoms.
Nat. Nanotechnol. ,10(7):577, 2015.[4] Tokuhisa Kawawaki and Yuichi Negishi. Gold nanoclusters as electrocatalysts for energy conversion.
Nanomaterials , 10(2):238, 2020.[5] Anne CH Jans, Xavier Caumes, and Joost NH Reek. Gold catalysis in (supra) molecular cages to controlreactivity and selectivity.
ChemCatChem , 11(1):287–297, 2019.[6] Seiji Yamazoe, Kiichirou Koyasu, and Tatsuya Tsukuda. Nonscalable oxidation catalysis of gold clusters.
Acc. Chem. Res. , 47(3):816–824, 2014.[7] Gao Li and Rongchao Jin. Atomically precise gold nanoclusters as new model catalysts.
Acc. Chem.Res. , 46(8):1749–1758, 2013.[8] H¨ulya Sak, Matthias Mawick, and Norbert Krause. Sustainable gold catalysis in water usingcyclodextrin-tagged nhc-gold complexes.
ChemCatChem , 11(23):5821–5829, 2019.139] Sujata Sengupta and Xiaodong Shi. Recent advances in asymmetric gold catalysis.
ChemCatChem , 2(6):609–619, 2010.[10] Stefan Vajda, Michael J Pellin, Jeffrey P Greeley, Christopher L Marshall, Larry A Curtiss, Gregory ABallentine, Jeffrey W Elam, Stephanie Catillon-Mucherie, Paul C Redfern, Faisal Mehmood, et al. Sub-nanometre platinum clusters as highly active and selective catalysts for the oxidative dehydrogenationof propane.
Nat. Mater. , 8(3):213–216, 2009.[11] Eric C Tyo, Chunrong Yin, Marcel Di Vece, Qiang Qian, Gihan Kwon, Sungsik Lee, Byeongdu Lee,Janae E DeBartolo, Sonke Seifert, Randall E Winans, et al. Oxidative dehydrogenation of cyclohexaneon cobalt oxide (co3o4) nanoparticles: The effect of particle size on activity and selectivity.
ACS Catal. ,2(11):2409–2423, 2012.[12] Zhiyong Guo, Chaoxian Xiao, Raghu V Maligal-Ganesh, Lin Zhou, Tian Wei Goh, Xinle Li, DanielTesfagaber, Andrew Thiel, and Wenyu Huang. Pt nanoclusters confined within metal–organic frameworkcavities for chemoselective cinnamaldehyde hydrogenation.
ACS Catal. , 4(5):1340–1348, 2014.[13] Jeff Greeley, Thomas F Jaramillo, Jacob Bonde, IB Chorkendorff, and Jens K Nørskov. Computationalhigh-throughput screening of electrocatalytic materials for hydrogen evolution.
Nat. Mater. , 5(11):909–913, 2006.[14] Sergio Tosoni and Gianfranco Pacchioni. Oxide-supported gold clusters and nanoparticles in catalysis:A computational chemistry perspective.
ChemCatChem , 11(1):73–89, 2019.[15] Peter Strasser, Qun Fan, Martin Devenney, W Henry Weinberg, Ping Liu, and Jens K Nørskov. Highthroughput experimental and theoretical predictive screening of materials- a comparative study of searchstrategies for new fuel cell anode catalysts.
J. Phys. Chem. B , 107(40):11013–11021, 2003.[16] John R Kitchin. Machine learning in catalysis.
Nat. Catal. , 1(4):230–232, 2018.[17] Ali Alavi, Peijun Hu, Thierry Deutsch, Pier Luigi Silvestrelli, and J¨urg Hutter. Co oxidation on pt(111): An ab initio density functional theory study.
Phys. Rev. Lett. , 80(16):3650, 1998.[18] Mark Anstrom, Nan-Yu Topsøe, and JA Dumesic. Density functional theory studies of mechanisticaspects of the scr reaction on vanadium oxide catalysts.
J. Catal. , 213(2):115–125, 2003.[19] Jens K Nørskov, Frank Abild-Pedersen, Felix Studt, and Thomas Bligaard. Density functional theoryin surface chemistry and catalysis.
Proc. Natl. Acad. Sci. U. S. A , 108(3):937–943, 2011.[20] Michael J Hostetler, Allen C Templeton, and Royce W Murray. Dynamics of place-exchange reactionson monolayer-protected gold cluster molecules.
Langmuir , 15(11):3782–3789, 1999.[21] Xiaoming Huang, Linwei Sai, Xue Jiang, and Jijun Zhao. Ground state structures, electronic and opticalproperties of medium-sized na n+(n= 9, 15, 21, 26, 31, 36, 41, 50 and 59) clusters from ab initio geneticalgorithm.
Eur. Phys. J. D , 67(2):43, 2013.[22] Jijun Zhao, Ruili Shi, Linwei Sai, Xiaoming Huang, and Yan Su. Comprehensive genetic algorithm forab initio global optimisation of clusters.
Mol. Simul. , 42(10):809–819, 2016.[23] Lixin Zhan, Jeff ZY Chen, Wing-Ki Liu, and SK Lai. Asynchronous multicanonical basin hoppingmethod and its application to cobalt nanoclusters.
J. Chem. Phys. , 122(24):244707, 2005.[24] Runhai Ouyang, Yu Xie, and De-en Jiang. Global minimization of gold clusters by combining neuralnetwork potentials and the basin-hopping method.
Nanoscale , 7(36):14817–14821, 2015.[25] Hyoung Gyu Kim, Si Kyung Choi, and Hyuck Mo Lee. New algorithm in the basin hopping monte carloto find the global minimum structure of unary and binary metallic nanoclusters.
J. Chem. Phys. , 128(14):144702, 2008. 1426] Alper Kinaci, Badri Narayanan, Fatih G Sen, Michael J Davis, Stephen K Gray, Subramanian KRSSankaranarayanan, and Maria KY Chan. Unraveling the planar-globular transition in gold nanoclustersthrough evolutionary search.
Sci. Rep. , 6:34974, 2016.[27] Badri Narayanan, Alper Kinaci, Fatih G Sen, Michael J Davis, Stephen K Gray, Maria KY Chan, andSubramanian KRS Sankaranarayanan. Describing the diverse geometries of gold from nanoclusters tobulk- a first-principles-based hybrid bond-order potential.
J. Phys. Chem. C , 120(25):13787–13800,2016.[28] Siva Chiriki, Shweta Jindal, and Satya S Bulusu. Neural network potentials for dynamics and thermo-dynamics of gold nanoparticles.
J. Chem. Phys. , 146(8):084314, 2017.[29] Shweta Jindal, Siva Chiriki, and Satya S Bulusu. Spherical harmonics based descriptor for neuralnetwork potentials: Structure and dynamics of au147 nanocluster.
J. Chem. Phys. , 146(20):204301,2017.[30] Nongnuch Artrith and Alexie M Kolpak. Understanding the composition and activity of electrocatalyticnanoalloys in aqueous solvents: A combination of dft and accurate neural network potentials.
NanoLett. , 14(5):2670–2676, 2014.[31] J¨org Behler. Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations.
Phys. Chem. Chem. Phys. , 13(40):17930–17955, 2011.[32] Ghanshyam Pilania, James E Gubernatis, and Turab Lookman. Multi-fidelity machine learning modelsfor accurate bandgap predictions of solids.
Comput. Mater. Sci. , 129:156–163, 2017.[33] Justin S Smith, Ben Nebgen, Nicholas Lubbers, Olexandr Isayev, and Adrian E Roitberg. Less is more:Sampling chemical space with active learning.
J. Chem. Phys. , 148(24):241733, 2018.[34] Linfeng Zhang, De-Ye Lin, Han Wang, Roberto Car, and E Weinan. Active learning of uniformlyaccurate interatomic potentials for materials simulation.
Phys. Rev. Mater. , 3(2):023804, 2019.[35] Jonathan Vandermause, Steven Torrisi, Simon Batzner, Alexie Kolpak, Boris Kozinsky, andJonathan Vandermause Team. Accelerating atomistic modelling with active learning. In
APS MeetingAbstracts , 2019.[36] Troy D Loeffler, Tarak K Patra, Henry Chan, Mathew Cherukara, and Subramanian KRS Sankara-narayanan. Active learning the potential energy landscape for water clusters from sparse training data.
J. Phys. Chem. C , 124(8):4907–4916, 2020.[37] Troy D Loeffler, Tarak K Patra, Henry Chan, and Subramanian KRS Sankaranarayanan. Active learninga coarse-grained neural network model for bulk water from sparse training data.
Mol. Syst. Des. Eng. ,2020.[38] Tarak K Patra, Troy D Loeffler, Henry Chan, Mathew J Cherukara, Badri Narayanan, and Subrama-nian KRS Sankaranarayanan. A coarse-grained deep neural network model for liquid water.
Appl. Phys.Lett. , 115(19):193101, 2019.[39] SM Foiles, MI Baskes, and Murray S Daw. Embedded-atom-method functions for the fcc metals cu, ag,au, ni, pd, pt, and their alloys.
Phys. Rev. B , 33(12):7983, 1986.[40] Murray S Daw and Michael I Baskes. Embedded-atom method: Derivation and application to impurities,surfaces, and other defects in metals.
Phys. Rev. B , 29(12):6443, 1984.[41] Jonathan PK Doye and David J Wales. Global minima for transition metal clusters described bysutton–chen potentials.
New J. Chem. , 22(7):733–744, 1998.[42] Xueguang Shao, Xiaomeng Liu, and Wensheng Cai. Structural optimization of silver clusters up to 80atoms with gupta and sutton-chen potentials.
J. Chem. Theory Comput. , 1(4):762–768, 2005.1543] M Backman, N Juslin, and K Nordlund. Bond order potential for gold.
Eur. Phys. J. B , 85(9):317,2012.[44] John A Keith, Donato Fantauzzi, Timo Jacob, and Adri CT Van Duin. Reactive forcefield for simulatinggold surfaces and nanoparticles.
Phys. Rev. B , 81(23):235404, 2010.[45] Li Xiao, Bethany Tollberg, Xiankui Hu, and Lichang Wang. Structural study of gold clusters.
J. Chem.Phys. , 124(11):114309, 2006.[46] Stefano A Serapian, Michael J Bearpark, and Fernando Bresme. The shape of au 8: gold leaf or goldnugget?
Nanoscale , 5(14):6445–6457, 2013.[47] Bryan R Goldsmith, Jacob Florian, Jin-Xun Liu, Philipp Gruene, Jonathan T Lyon, David M Rayner,Andr´e Fielicke, Matthias Scheffler, and Luca M Ghiringhelli. Two-to-three dimensional transition inneutral gold clusters: The crucial role of van der waals interactions and temperature.
Phys. Rev. Mater. ,3(1):016002, 2019.[48] Nongnuch Artrith and Alexander Urban. An implementation of artificial neural-network potentials foratomistic materials simulations: Performance for TiO2.
Comput. Mater. Sci. , 114:135 – 150, 2016. ISSN0927-0256.[49] Troy D Loeffler. Molecular monte carlo code. https://github.com/mrnucleation/ClassyMC , 2020.[50] Georg Kresse and J¨urgen Furthm¨uller. Efficiency of ab-initio total energy calculations for metals andsemiconductors using a plane-wave basis set.
Comput. Mater. Sci. , 6(1):15–50, 1996.[51] G. Kresse and J. Furthm¨uller. Efficient iterative schemes for ab initio total-energy calculations using aplane-wave basis set.
Phys. Rev. B , 54:11169–11186, Oct 1996. doi: 10.1103/PhysRevB.54.11169.[52] Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, StephenDacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The materialsproject: A materials genome approach to accelerating materials innovation.
APL Mater. , 1(1):011002,2013.[53] SKR Patil, SV Khare, Blair Richard Tuttle, JK Bording, and S Kodambaka. Mechanical stability ofpossible structures of ptn investigated using first-principles calculations.
Phys. Rev. B , 73(10):104118,2006.[54] Sukriti Manna, Geoff L Brennecka, Vladan Stevanovi´c, and Cristian V Ciobanu. Tuning the piezoelectricand mechanical properties of the aln system via alloying with yn and bn.
J. Appl. Phys. , 122(10):105101,2017.[55] Dong Wu, Yachao Chen, Sukriti Manna, Kevin Talley, Andriy Zakutayev, Geoff L Brennecka, Cristian VCiobanu, Paul Constantine, and Corinne E Packard. Characterization of elastic modulus across the(al 1–x sc x) n system using dft and substrate-effect-corrected nanoindentation.
IEEE Trans. SonicsUltrason. , 65(11):2167–2175, 2018.[56] Sukriti Manna, Prashun Gorai, Geoff L Brennecka, Cristian V Ciobanu, and Vladan Stevanovi´c. Largepiezoelectric response of van der waals layered solids.
J. Mater. Chem. C , 6(41):11035–11044, 2018.[57] Sukriti Manna, Kevin R Talley, Prashun Gorai, John Mangum, Andriy Zakutayev, Geoff L Brennecka,Vladan Stevanovi´c, and Cristian V Ciobanu. Enhanced piezoelectric response of aln via crn alloying.
Phys. Rev. Appl. , 9(3):034026, 2018.[58] Robert W McKinney, Prashun Gorai, Sukriti Manna, Eric Toberer, and Vladan Stevanovi´c. Ionic vs.van der waals layered materials: identification and comparison of elastic anisotropy.
J. Mater. Chem.A , 6(32):15828–15838, 2018. 1659] Sukriti Manna, Geoff Brennecka, Vladan Stevanovic, and Cristian V Ciobanu. Tuning the piezoelectricand mechanical properties of the aln system via alloying with yn and bn, April 25 2019. US PatentApp. 16/158,826.[60] J¨org Behler. Atom-centered symmetry functions for constructing high-dimensional neural network po-tentials.
J. Chem. Phys. , 134(7):074106, 2011.[61] Jose Pujol. The solution of nonlinear inverse problems and the levenberg-marquardt method.
Geophysics ,72(4):W1–W16, 2007.[62] Vijander Singh, Indra Gupta, and HO Gupta. Ann-based estimator for distillation using levenberg–marquardt approach.
Eng. Appl. Artif. Intell , 20(2):249–259, 2007.[63] Steven O Nielsen. Nested sampling in the canonical ensemble: Direct calculation of the partitionfunction from nvt trajectories.
J. Chem. Phys. , 139(12):124104, 2013.[64] Anubhav Jain, Geoffroy Hautier, Charles J Moore, Shyue Ping Ong, Christopher C Fischer, Tim Mueller,Kristin A Persson, and Gerbrand Ceder. A high-throughput infrastructure for density functional theorycalculations.
Comput. Mater. Sci. , 50(8):2295–2310, 2011.[65] AP Sutton and J Chen. Long-range finnis–sinclair potentials.
Philos. Mag. Lett. , 61(3):139–146, 1990.[66] T Cagin, Y Qi, H Li, Y Kimura, H Ikeda, WL Johnson, and WA Goddard. Calculation of thermal,mechanical and transport properties of model glass formers. In
Bulk Metallic Glasses MRS Symp. Ser ,volume 554, 1999.[67] BA Collings, K Athanassenas, D Lacombe, DM Rayner, and PA Hackett. Optical absorption spectra ofau7, au9, au11, and au13, and their cations: Gold clusters with 6, 7, 8, 9, 10, 11, 12, and 13 s-electrons.
J. Chem. Phys. , 101(5):3506–3513, 1994.[68] Mathis Gruber, Georg Heimel, Lorenz Romaner, Jean-Luc Br´edas, and Egbert Zojer. First-principlesstudy of the geometric and electronic structure of au 13 clusters: Importance of the prism motif.
Phys.Rev. B , 77(16):165411, 2008.[69] Jinlan Wang, Guanghou Wang, and Jijun Zhao. Density-functional study of au n (n= 2-20) clusters:Lowest-energy structures and electronic properties.
Phys. Rev. B , 66(3):035418, 2002.[70] Jordan E Vincent, Jeongnim Kim, and Richard M Martin. Quantum monte carlo calculations of theoptical gaps of ge nanoclusters using core-polarization potentials.
Phys. Rev. B , 75(4):045302, 2007.[71] Gustavo E Scuseria and Henry F Schaefer III. Is coupled cluster singles and doubles (ccsd) morecomputationally intensive than quadratic configuration interaction (qcisd)?