Vladimir Hnizdo

Nearest Neighbor Estimates of Entropy

SYNOPTIC ABSTRACT Motivated by the problems in molecular sciences, we introduce new nonparametric estimators of entropy which are based on the kth nearest neighbor distances between the n sample points, where k (< n – 1) is a fixed positive integer. These provide competing estimators to an estimator proposed by Kozachenko and Leonenko (1987), which is based on the first nearest neighbor distances of the sample points. These estimators are helpful in the evaluation of entropies of random vectors. We establish the asymptotic unbiasedness and consistency of the proposed estimators. For some standard distributions, we also investigate their performance for finite sample sizes using Monte Carlo simulations. The proposed estimators are applied to estimate the entropy of internal rotation in the methanol molecule, which can be characterized by a one-dimensional random vector, and of diethyl ether, which is described by a four-dimensional random vector.

Efficient calculation of configurational entropy from molecular simulations by combining the mutual-information expansion and nearest-neighbor methods†‡

Changes in the configurational entropies of molecules make important contributions to the free energies of reaction for processes such as protein‐folding, noncovalent association, and conformational change. However, obtaining entropy from molecular simulations represents a long‐standing computational challenge. Here, two recently introduced approaches, the nearest‐neighbor (NN) method and the mutual‐information expansion (MIE), are combined to furnish an efficient and accurate method of extracting the configurational entropy from a molecular simulation to a given order of correlations among the internal degrees of freedom. The resulting method takes advantage of the strengths of each approach. The NN method is entirely nonparametric (i.e., it makes no assumptions about the underlying probability distribution), its estimates are asymptotically unbiased and consistent, and it makes optimum use of a limited number of available data samples. The MIE, a systematic expansion of entropy in mutual information terms of increasing order, provides a well‐characterized approximation for lowering the dimensionality of the numerical problem of calculating the entropy of a high‐dimensional system. The combination of these two methods enables obtaining well‐converged estimations of the configurational entropy that capture many‐body correlations of higher order than is possible with the simple histogramming that was used in the MIE method originally. The combined method is tested here on two simple systems: an idealized system represented by an analytical distribution of six circular variables, where the full joint entropy and all the MIE terms are exactly known, and the R,S stereoisomer of tartaric acid, a molecule with seven internal‐rotation degrees of freedom for which the full entropy of internal rotation has been already estimated by the NN method. For these two systems, all the expansion terms of the full MIE of the entropy are estimated by the NN method and, for comparison, the MIE approximations up to third order are also estimated by simple histogramming. The results indicate that the truncation of the MIE at the two‐body level can be an accurate, computationally nondemanding approximation to the configurational entropy of anharmonic internal degrees of freedom. If needed, higher‐order correlations can be estimated reliably by the NN method without excessive demands on the molecular‐simulation sample size and computing time.

Nearest-neighbor nonparametric method for estimating the configurational entropy of complex molecules.

A method for estimating the configurational (i.e., non‐kinetic) part of the entropy of internal motion in complex molecules is introduced that does not assume any particular parametric form for the underlying probability density function. It is based on the nearest‐neighbor (NN) distances of the points of a sample of internal molecular coordinates obtained by a computer simulation of a given molecule. As the method does not make any assumptions about the underlying potential energy function, it accounts fully for any anharmonicity of internal molecular motion. It provides an asymptotically unbiased and consistent estimate of the configurational part of the entropy of the internal degrees of freedom of the molecule. The NN method is illustrated by estimating the configurational entropy of internal rotation of capsaicin and two stereoisomers of tartaric acid, and by providing a much closer upper bound on the configurational entropy of internal rotation of a pentapeptide molecule than that obtained by the standard quasi‐harmonic method. As a measure of dependence between any two internal molecular coordinates, a general coefficient of association based on the information‐theoretic quantity of mutual information is proposed. Using NN estimates of this measure, statistical clustering procedures can be employed to group the coordinates into clusters of manageable dimensions and characterized by minimal dependence between coordinates belonging to different clusters.

Thermodynamic and Differential Entropy under a Change of Variables

The differential Shannon entropy of information theory can change under a change of variables (coordinates), but the thermodynamic entropy of a physical system must be invariant under such a change. This difference is puzzling, because the Shannon and Gibbs entropies have the same functional form. We show that a canonical change of variables can, indeed, alter the spatial component of the thermodynamic entropy just as it alters the differential Shannon entropy. However, there is also a momentum part of the entropy, which turns out to undergo an equal and opposite change when the coordinates are transformed, so that the total thermodynamic entropy remains invariant. We furthermore show how one may correctly write the change in total entropy for an isothermal physical process in any set of spatial coordinates.

Correlation as a determinant of configurational entropy in supramolecular and protein systems.

For biomolecules in solution, changes in configurational entropy are thought to contribute substantially to the free energies of processes like binding and conformational change. In principle, the configurational entropy can be strongly affected by pairwise and higher-order correlations among conformational degrees of freedom. However, the literature offers mixed perspectives regarding the contributions that changes in correlations make to changes in configurational entropy for such processes. Here we take advantage of powerful techniques for simulation and entropy analysis to carry out rigorous in silico studies of correlation in binding and conformational changes. In particular, we apply information-theoretic expansions of the configurational entropy to well-sampled molecular dynamics simulations of a model host–guest system and the protein bovine pancreatic trypsin inhibitor. The results bear on the interpretation of NMR data, as they indicate that changes in correlation are important determinants of entropy changes for biologically relevant processes and that changes in correlation may either balance or reinforce changes in first-order entropy. The results also highlight the importance of main-chain torsions as contributors to changes in protein configurational entropy. As simulation techniques grow in power, the mathematical techniques used here will offer new opportunities to answer challenging questions about complex molecular systems.

Generalized second-order partial derivatives of 1/r

The generalized second-order partial derivatives of 1/r, where r is the radial distance in three dimensions (3D), are obtained using a result of the potential theory of classical analysis. Some non-spherical-regularization alternatives to the standard spherical-regularization expression for the derivatives are derived. The utility of a spheroidal-regularization expression is illustrated on an example from classical electrodynamics.

Estimation of the absolute internal-rotation entropy of molecules with two torsional degrees of freedom from stochastic simulations.

A method of statistical estimation is applied to the problem of evaluating the absolute entropy of internal rotation in a molecule with two torsional degrees of freedom. The configurational part of the entropy is obtained as that of the joint probability density of an arbitrary form represented by a two‐dimensional Fourier series, the coefficients of which are statistically estimated using a sample of the torsional angles of the molecule obtained by a stochastic simulation. The internal rotors in the molecule are assumed to be attached to a common frame, and their reduced moments of inertia are initially calculated as functions of the two torsional angles, but averaged over all the remaining internal degrees of freedom using the stochastic‐simulation sample of the atomic configurations of the molecule. The torsional‐angle dependence of the reduced moments of inertia can be also averaged out, and the absolute internal‐rotation entropy of the molecule is obtained in a good approximation as the sum of the configurational entropy and a kinetic contribution fully determined by the averaged reduced moments of inertia. The method is illustrated using Monte Carlo simulations of isomers of stilbene and halogenated derivatives of propane. The two torsional angles in cis‐stilbene are found to be much more strongly correlated than those in trans‐stilbene, while the degree of the angular correlation in propane increases strongly on substitution of hydrogen atoms with chlorine.

Statistical thermodynamics of internal rotation in a hindering potential of mean force obtained from computer simulations.

A method of statistical estimation is applied to the problem of one‐dimensional internal rotation in a hindering potential of mean force. The hindering potential, which may have a completely general shape, is expanded in a Fourier series, the coefficients of which are estimated by fitting an appropriate statistical–mechanical distribution to the random variable of internal rotation angle. The function of reduced moment of inertia of an internal rotation is averaged over the thermodynamic ensemble of atomic configurations of the molecule obtained in stochastic simulations. When quantum effects are not important, an accurate estimate of the absolute internal rotation entropy of a molecule with a single rotatable bond is obtained. When there is more than one rotatable bond, the “marginal” statistical–mechanical properties corresponding to a given internal rotational degree of freedom are educed. The method is illustrated using Monte Carlo simulations of two public health relevant halocarbon molecules, each having a single internal‐rotation degree of freedom, and a molecular dynamics simulation of an immunologically relevant polypeptide, in which several dihedral angles are analyzed.

Comment on ‘Electromagnetic force on a moving dipole’

Using the Lagrangian formalism, the force on a moving dipole derived by Kholmetskii et al (2011 Eur. J. Phys. 32 873–81) is found to be missing some important terms.

Potentials of a uniformly moving point charge in the Coulomb gauge

The Coulomb-gauge vector potential of a uniformly moving point charge is obtained by calculating the gauge function for the transformation between the Lorenz and Coulomb gauges. The expression obtained for the difference between the vector potentials in the two gauges is shown to satisfy a Poisson equation to which the inhomogeneous wave equation for this quantity can be reduced. The right-hand side of the Poisson equation involves an important but easily overlooked delta-function term that arises from a second-order partial derivative of the Coulomb potential of a point charge.