Paolo Sibani

Facts, Conjectures, and Improvements for Simulated Annealing

From the Publisher: Simulated annealing has proved to be an easy and reliable method for finding optimal values of a problem in cases where there is no road map to possible solutions. Facts, Conjectures, and Improvements for Simulated Annealing offers an introduction to this topic for novices and provides an informative review of the area for the more expert reader. This book brings together for the first time many of the theoretical foundations for improvements to algorithms for global optimization that until now existed only in scattered research articles. The method described in this book operates by simulating the cooling of a (usually fictitious) physical system whose possible energies correspond to the values of the objective function being minimized. The analogy works because physical systems occupy only states with the lowest energy as the temperature is lowered to absolute zero. his book is suitable for advanced undergraduate and graduate students and for professionals in a wide variety of subject areas: bioinformatics, chemistry, computer science, engineering, finance, geology, mathematics, and physics.

Evolution in complex systems

What features characterize complex system dynamics? Power laws and scale invariance of fluctuations are often taken as the hallmarks of complexity, drawing on analogies with equilibrium critical phenomena. Here we argue that slow, directed dynamics, during which the system’s properties change significantly, is fundamental. The underlying dynamics is related to a slow, decelerating but spasmodic release of an intrinsic strain or tension. Time series of a number of appropriate observables can be analyzed to confirm this effect. The strain arises from local frustration. As the strain is released through “quakes,” some system variable undergoes record statistics with accompanying log-Poisson statistics for the quake event times. We demonstrate these phenomena via two very different systems: a model of magnetic relaxation in type II superconductors and the Tangled Nature model of evolutionary ecology and show how quantitative indications of aging can be found.

Comparing extremal and thermal explorations of energy landscapes

Abstract.Using a non-thermal local search, called Extremal Optimization (EO), in conjunction with a recently developed scheme for classifying the valley structure of complex systems, we analyze a short-range spin glass. In comparison with earlier studies using a thermal algorithm with detailed balance, we determine which features of the landscape are algorithm dependent and which are inherently geometrical. Apparently a characteristic for any local search in complex energy landscapes, the time series of successive energy records found by EO is also characterized approximately by a Poisson statistic with logarithmic time arguments. Differences in the results provide additional insights into the performance of EO. In contrast with a thermal search, the extremal search visits dramatically higher energies while returning to more widely separated low-energy configurations. Two important properties of the energy landscape are independent of either algorithm: first, to find lower energy records, progressively higher energy barriers need to be overcome. Second, the Hamming distance between two consecutive low-energy records is linearly related to the height of the intervening barrier.

Relaxation and aging in spin glasses and other complex systems

We describe how a hierachical model of spin glass relaxation can display aging behaviour, similarly to what is found in spin glasses and other complex systems out of thermodynamic equilibrium. Since we deal with a nonequilibrium situation, the usualF(luctuation)D(issipation)T(heory) does not apply. We therefore derive a general relation between the linear response function and the non equilibrium propagator of the unperturbed system. The relation is shown to be very similar to the equilibrium FDT under certain conditions, which one can reasonably assume for spin glass systems. Having thus related the linear response of the system to a small external field to the autocorrelation function of the magnetization, we calculate the latter quantity by a master equation on a set of states which have the topology of a tree. The model can reproduce the main qualitative features of theZ(ero)F(ield)C(ooled) spin glass experiments, i.e. the maximum in the logarithmic time derivative of the magnetization, with only two free parameters.

Local state space geometry and thermal relaxation in complex landscapes: the spin-glass case

A simple geometrical characterization of configuration space neighborhoods of local energy minima in spin glass landscapes is found by exhaustive search. Combined with previous Monte Carlo investigations of thermal domain growth, it allows a discussion of the connection between real and configuration space descriptions of low temperature relaxational dynamics. We argue that the part of state-space corresponding to a single growing domain is adequately modeled by a hierarchically organized set of states and that thermal (meta)stability in spin glasses is related to the nearly exponential local density of states present within each trap.

Optimal ensemble size for parallel implementations of simulated annealing

Abstract We determine the optimal ensemble size for a simulated annealing problem based on assumptions about scaling properties of the system dynamics and of the density of states in the low energy regime. The derivations indicate the optimal annealing time for any one ensemble member, thereby providing a stopping criterion and an explanation for the “brick wall effect”.

Aging and relaxation dynamics in free-energy landscapes with multiple minima

Abstract We consider the stochastic dynamics of a system thermally relaxing in a free-energy landscape with multiple attractors, and show that lack of translational homogeneity in this landscape leads to aging effects, e.g. to the dependence of the susceptibilities on the time elapsed from a thermal quench to the imposition of the probing field. We then prove an inequality between response and correlation which generalizes the fluctuation dissipation theorem to a situation far from thermodynamical equilibrium. As an application and a check we specialize our formalism in a way which we suggest is appropriate for spin-glass systems: we assume a hierarchical organization of the landscape, and find aging behavior in the response curves in good agreement with relevant experimental data. We finally conclude with a summary and a brief discussion of different approaches to slow relaxation in complex systems.

Domain growth and thermal relaxation in spin glasses

The low-temperature growth of correlated domains in nearest neighbor Gaussian Ising spin glases in 2, 3 and 4 dimensions is studied by two different Monte Carlo techniques. The first is patterned on the ‘lid’ method previously proposed by the authors. The simulation begins at a low-energy ‘reference’ state which is previously found by annealing, and studies how the system diffuses away from this configuration. The local energy minimum configurations reached in time t during the relaxation at temperature T are analyzed by identifying all the connected clusters of spins which have reversed their initial orientation. In particular, the growth of the average cluster volume is examined. The second approach, which is due to Huse, considers the way in which two replicas of the systems approach a low-energy configuration starting from random and uncorrelated high-energy states. Their time coarse grained magnetizations are compared, yielding a measure of the linear cluster size, again as a function of t and T. All our low-temperature data can be scaled on a master curve, whose abscissa b(t, T) is the line in the t, T plane corresponding to constant cluster volume. The region in which this parametrization exists corresponds to the usual thermodynamical low T phases in 3 and 4D. Within the region the form of the growth laws depends on the dimensionality of the system, and on the type of relaxation. For our method we also find a strong and systematic dependence of the growth laws on the annealing time spent to identify the reference state, which is akin to the aging dependence of the linear response: the longer the annealing time, the slower is the growth.

Entropy in the Tangled Nature Model of Evolution

Applications of entropy principles to evolution and ecology are of tantamount importance given the central role spatiotemporal structuring plays in both evolution and ecological succession. We obtain here a qualitative interpretation of the role of entropy in evolving ecological systems. Our interpretation is supported by mathematical arguments using simulation data generated by the Tangled Nature Model (TNM), a stochastic model of evolving ecologies. We define two types of configurational entropy and study their empirical time dependence obtained from the data. Both entropy measures increase logarithmically with time, while the entropy per individual decreases in time, in parallel with the growth of emergent structures visible from other aspects of the simulation. We discuss the biological relevance of these entropies to describe niche space and functional space of ecosystems, as well as their use in characterizing the number of taxonomic configurations compatible with different niche partitioning and functionality. The TNM serves as an illustrative example of how to calculate and interpret these entropies, which are, however, also relevant to real ecosystems, where they can be used to calculate the number of functional and taxonomic configurations that an ecosystem can realize.

Temperature dependence of effective fluctuation time scales in spin glasses

Using a series of fast-cooling protocols we have probed aging effects in the spin-glass state as a function of temperature. Analyzing the logarithmic decay found at very long-time scales within a simple phenomenological barrier model leads to the extraction of an effective fluctuation time scale of the system at a particular temperature. This is the smallest dynamical time-scale defining a lower cutoff in a hierarchical description of the dynamics. We find that this fluctuation time scale, which is approximately equal to atomic spin-fluctuation time scales near the transition temperature, follows a generalized Arrhenius law. We discuss the hypothesis that, upon cooling to a measuring temperature within the spin-glass state, there is a range of dynamically inequivalent configurations in which the system can be trapped, and check within a numerical barrier model simulation, that this leads to subaging behavior in scaling aged thermoremanent magnetization decay curves, as recently discussed theoretically [P. Sibani and G. G. Kenning, Phys. Rev. E 81, 011108 (2010)].