Stuart J. Davie

Prediction of Intramolecular Polarization of Aromatic Amino Acids Using Kriging Machine Learning.

Present computing power enables novel ways of modeling polarization. Here we show that the machine learning method kriging accurately captures the way the electron density of a topological atom responds to a change in the positions of the surrounding atoms. The success of this method is demonstrated on the four aromatic amino acids histidine, phenylalanine, tryptophan, and tyrosine. A new technique of varying training set sizes to vastly reduce training times while maintaining accuracy is described and applied to each amino acid. Each amino acid has its geometry distorted via normal modes of vibration over all local energy minima in the Ramachandran map. These geometries are then used to train the kriging models. Total electrostatic energies predicted by the kriging models for previously unseen geometries are compared to the true energies, yielding mean absolute errors of 2.9, 5.1, 4.2, and 2.8 kJ mol(-1) for histidine, phenylalanine, tryptophan, and tyrosine, respectively.

Optimization Algorithms in Optimal Predictions of Atomistic Properties by Kriging

The machine learning method kriging is an attractive tool to construct next-generation force fields. Kriging can accurately predict atomistic properties, which involves optimization of the so-called concentrated log-likelihood function (i.e., fitness function). The difficulty of this optimization problem quickly escalates in response to an increase in either the number of dimensions of the system considered or the size of the training set. In this article, we demonstrate and compare the use of two search algorithms, namely, particle swarm optimization (PSO) and differential evolution (DE), to rapidly obtain the maximum of this fitness function. The ability of these two algorithms to find a stationary point is assessed by using the first derivative of the fitness function. Finally, the converged position obtained by PSO and DE is refined through the limited-memory Broyden-Fletcher-Goldfarb-Shanno bounded (L-BFGS-B) algorithm, which belongs to the class of quasi-Newton algorithms. We show that both PSO and DE are able to come close to the stationary point, even in high-dimensional problems. They do so in a reasonable amount of time, compared to that with the Newton and quasi-Newton algorithms, regardless of the starting position in the search space of kriging hyperparameters. The refinement through L-BFGS-B is able to give the position of the maximum with whichever precision is desired.

Incorporation of local structure into kriging models for the prediction of atomistic properties in the water decamer

Machine learning algorithms have been demonstrated to predict atomistic properties approaching the accuracy of quantum chemical calculations at significantly less computational cost. Difficulties arise, however, when attempting to apply these techniques to large systems, or systems possessing excessive conformational freedom. In this article, the machine learning method kriging is applied to predict both the intra‐atomic and interatomic energies, as well as the electrostatic multipole moments, of the atoms of a water molecule at the center of a 10 water molecule (decamer) cluster. Unlike previous work, where the properties of small water clusters were predicted using a molecular local frame, and where training set inputs (features) were based on atomic index, a variety of feature definitions and coordinate frames are considered here to increase prediction accuracy. It is shown that, for a water molecule at the center of a decamer, no single method of defining features or coordinate schemes is optimal for every property. However, explicitly accounting for the structure of the first solvation shell in the definition of the features of the kriging training set, and centring the coordinate frame on the atom‐of‐interest will, in general, return better predictions than models that apply the standard methods of feature definition, or a molecular coordinate frame.

FEREBUS: Highly parallelized engine for kriging training

FFLUX is a novel force field based on quantum topological atoms, combining multipolar electrostatics with IQA intraatomic and interatomic energy terms. The program FEREBUS calculates the hyperparameters of models produced by the machine learning method kriging. Calculation of kriging hyperparameters (θ and p) requires the optimization of the concentrated log‐likelihood L̂(θ,p) . FEREBUS uses Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithms to find the maximum of L̂(θ,p) . PSO and DE are two heuristic algorithms that each use a set of particles or vectors to explore the space in which L̂(θ,p) is defined, searching for the maximum. The log‐likelihood is a computationally expensive function, which needs to be calculated several times during each optimization iteration. The cost scales quickly with the problem dimension and speed becomes critical in model generation. We present the strategy used to parallelize FEREBUS, and the optimization of L̂(θ,p) through PSO and DE. The code is parallelized in two ways. MPI parallelization distributes the particles or vectors among the different processes, whereas the OpenMP implementation takes care of the calculation of L̂(θ,p) , which involves the calculation and inversion of a particular matrix, whose size increases quickly with the dimension of the problem. The run time shows a speed‐up of 61 times going from single core to 90 cores with a saving, in one case, of ∼98% of the single core time. In fact, the parallelization scheme presented reduces computational time from 2871 s for a single core calculation, to 41 s for 90 cores calculation.

The free energy of expansion and contraction: Treatment of arbitrary systems using the Jarzynski equality

Thermodynamic integration, free energy perturbation, and slow change techniques have long been utilised in the calculation of free energy differences between two states of a system that has undergone some transformation. With the introduction of the Jarzynski equality and the Crooks relation, new approaches are possible. This paper investigates an important phenomenon - systems undergoing a change in volume/density - and derives both the Jarzynski equality and Crooks relation of such systems using a statistical mechanical approach. These results apply to systems with arbitrary particle interactions and densities. The application of this approach to the expansion/compression of particles confined within a vessel with a piston and within a periodic system is considered.

Kriging atomic properties with a variable number of inputs

A new force field called FFLUX uses the machine learning technique kriging to capture the link between the properties (energies and multipole moments) of topological atoms (i.e., output) and the coordinates of the surrounding atoms (i.e., input). Here we present a novel, general method of applying kriging to chemical systems that do not possess a fixed number of (geometrical) inputs. Unlike traditional kriging methods, which require an input system to be of fixed dimensionality, the method presented here can be readily applied to molecular simulation, where an interaction cutoff radius is commonly used and the number of atoms or molecules within the cutoff radius is not constant. The method described here is general and can be applied to any machine learning technique that normally operates under a fixed number of inputs. In particular, the method described here is also useful for interpolating methods other than kriging, which may suffer from difficulties stemming from identical sets of inputs corresponding to different outputs or input biasing. As a demonstration, the new method is used to predict 54 energetic and electrostatic properties of the central water molecule of a set of 5000, 4 Å radius water clusters, with a variable number of water molecules. The results are validated against equivalent models from a set of clusters composed of a fixed number of water molecules (set to ten, i.e., decamers) and against models created by using a naïve method of treating the variable number of inputs problem presented. Results show that the 4 Å water cluster models, utilising the method presented here, return similar or better kriging models than the decamer clusters for all properties considered and perform much better than the truncated models.

Applicability of optimal protocols and the Jarzynski equality

The Jarzynski equality is a well-known and widely used identity, relating the free energy difference between two states of a system to the work done over some arbitrary, nonequilibrium transformation between the two states. Despite being valid for both stochastic and deterministic systems, we show that the optimal transformation protocol for the deterministic case seems to differ from that predicated from an analysis of the stochastic dynamics. In addition, it is shown that for certain situations, more dissipative processes can sometimes lead to better numerical results for the free energy differences.

The accuracy of ab initio calculations without ab initio calculations for charged systems: Kriging predictions of atomistic properties for ions in aqueous solutions

Using the machine learning method kriging, we predict the energies of atoms in ion-water clusters, consisting of either Cl- or Na+ surrounded by a number of water molecules (i.e., without Na+Cl- interaction). These atomic energies are calculated following the topological energy partitioning method called Interacting Quantum Atoms (IQAs). Kriging predicts atomic properties (in this case IQA energies) by a model that has been trained over a small set of geometries with known property values. The results presented here are part of the development of an advanced type of force field, called FFLUX, which offers quantum mechanical information to molecular dynamics simulations without the limiting computational cost of ab initio calculations. The results reported for the prediction of the IQA components of the energy in the test set exhibit an accuracy of a few kJ/mol, corresponding to an average error of less than 5%, even when a large cluster of water molecules surrounding an ion is considered. Ions represent an important chemical system and this work shows that they can be correctly taken into account in the framework of the FFLUX force field.

Free Energy Calculations with Reduced Potential Cutoff Radii

The Jarzynski Equality, the Crooks Fluctuation Theorem, and the Maximum Likelihood Estimator use a nonequilibrium approach for the determination of free energy differences due to a change in the state of a system. Here, this approach is used in combination with a novel transformation algorithm to increase computational efficiency in simulations with interacting particles, without losing accuracy. The algorithm is shown to work well for a Lennard-Jones fluid undergoing a change in density over three very different density ranges, and for the systems considered the algorithm demonstrates computational savings of up to approximately 90%. The results obtained directly from the Jarzynski Equality and from the Maximum Likelihood Estimator are also compared.