Condensed Matter

Many sensory pathways in the brain rely on sparsely active populations of neurons downstream from the input stimuli. The biological reason for the occurrence of expanded structure in the brain is unclear, but may be because expansion can increase the expressive power of a neural network. In this work, we show that expanding a neural network can improve its generalization performance even in cases in which the expanded structure is pruned after the learning period. To study this setting we use a teacher-student framework where a perceptron teacher network generates labels which are corrupted with small amounts of noise. We then train a student network that is structurally matched to the teacher and can achieve optimal accuracy if given the teacher's synaptic weights. We find that sparse expansion of the input of a student perceptron network both increases its capacity and improves the generalization performance of the network when learning a noisy rule from a teacher perceptron when these expansions are pruned after learning. We find similar behavior when the expanded units are stochastic and uncorrelated with the input and analyze this network in the mean field limit. We show by solving the mean field equations that the generalization error of the stochastic expanded student network continues to drop as the size of the network increases. The improvement in generalization performance occurs despite the increased complexity of the student network relative to the teacher it is trying to learn. We show that this effect is closely related to the addition of slack variables in artificial neural networks and suggest possible implications for artificial and biological neural networks.

We present a random matrix approach to study general vibrational properties of stable amorphous solids with translational invariance using the correlated Wishart ensemble. Within this approach, both analytical and numerical methods can be applied. Using the random matrix theory, we found the analytical form of the vibrational density of states and the dynamical structure factor. We demonstrate the presence of the Ioffe-Regel crossover between low-frequency propagating phonons and diffusons at higher frequencies. The reduced vibrational density of states shows the boson peak, which frequency is close to the Ioffe-Regel crossover. We also present a simple numerical random matrix model with finite interaction radius, which properties rapidly converges to the analytical results with increasing the interaction radius. For fine interaction radius, the numerical model demonstrates the presence of the quasilocalized vibrations with a power-law low-frequency density of states.

We investigate a possible relation between frustration and phase-transition points in spin glasses. The relation is represented as a condition of the number of frustrated plaquettes in the lattice at phase-transition points at zero temperature and was reported to provide very close points to the phase-transition points for several lattices. Although there has been no proof of the relation, the good correspondence in several lattices suggests the validity of the relation and some important role of frustration in the phase transitions. To examine the relation further, we present a natural extension of the relation to diluted lattices and verify its effectiveness for bond-diluted square lattices. We then confirm that the resulting points are in good agreement with the phase-transition points in a wide range of dilution rate. Our result supports the suggestion from the previous work for non-diluted lattices on the importance of frustration to the phase transition of spin glasses.

Metaheuristics, as the simulated annealing used in the optimization of disordered systems, goes beyond physics, and the travelling salesman is a paradigmatic NP-complete problem that allows to infer important theoretical properties of the algorithm in different random environments. Many versions of the algorithm are explored in the literature, but so far the effects of the statistical distribution of the coordinates of the cities on the performance of the algorithm has been neglected. We propose a simple way to explore this aspect by analyzing the performance of a standard version of the simulated annealing (geometric cooling) in correlated systems with a simple and useful method based on a linear combination of independent random variables. Our results suggest that performance depends on the shape of the statistical distribution of the coordinates but not necessarily on its variance corroborated by the cases of uniform and normal distributions. On the other hand, a study with different power laws (different decay exponents) for the coordinates always produces different performances. We show that the performance of the simulated annealing, even in its best version, is not improved when the distribution of the coordinates does not have the first moment. However, surprisingly, we still observe improvements in situations where the second moment is not defined but not where the first one is not defined. Finite size scaling, fits, and universal laws support all of our results. In addition our study show when the cost must be scaled.

Kinetic Ising models are powerful tools for studying the non-equilibrium dynamics of complex systems. As their behavior is not tractable for large networks, many mean-field methods have been proposed for their analysis, each based on unique assumptions about the system's temporal evolution. This disparity of approaches makes it challenging to systematically advance mean-field methods beyond previous contributions. Here, we propose a unifying framework for mean-field theories of asymmetric kinetic Ising systems from an information geometry perspective. The framework is built on Plefka expansions of a system around a simplified model obtained by an orthogonal projection to a sub-manifold of tractable probability distributions. This view not only unifies previous methods but also allows us to develop novel methods that, in contrast with traditional approaches, preserve the system's correlations. We show that these new methods can outperform previous ones in predicting and assessing network properties near maximally fluctuating regimes.

The expressive power of artificial neural networks crucially depends on the nonlinearity of their activation functions. Though a wide variety of nonlinear activation functions have been proposed for use in artificial neural networks, a detailed understanding of their role in determining the expressive power of a network has not emerged. Here, we study how activation functions affect the storage capacity of treelike two-layer networks. We relate the boundedness or divergence of the capacity in the infinite-width limit to the smoothness of the activation function, elucidating the relationship between previously studied special cases. Our results show that nonlinearity can both increase capacity and decrease the robustness of classification, and provide simple estimates for the capacity of networks with several commonly used activation functions. Furthermore, they generate a hypothesis for the functional benefit of dendritic spikes in branched neurons.

Disconnectivity graphs are used to visualize the minima and the lowest energy barriers between the minima of complex systems. They give an easy and intuitive understanding of the underlying energy landscape and, as such, are excellent tools for understanding the complexity involved in finding low-lying or global minima of such systems. We have developed a classification scheme that categorizes highly-degenerate minima of spin glasses based on similarity and accessibility of the individual states. This classification allows us to condense the information pertained in different dales of the energy landscape to a single representation using color to distinguish its type and a bar chart to indicate the average size of the dales at their respective energy levels. We use this classification to visualize disconnectivity graphs of small representations of different tile-planted models of spin glasses. An analysis of the results shows that different models have distinctly different features in the total number of minima, the distribution of the minima with respect to the ground state, the barrier height and in the occurrence of the different types of minimum energy dales.

We review 15 years of theoretical and experimental work on the non-linear response of glassy systems. We argue that an anomalous growth of the peak value of non-linear susceptibilities is a signature of growing "amorphous order" in the system, with spin-glasses as a case in point. Experimental results on supercooled liquids are fully compatible with the RFOT prediction of compact "glassites" of increasing volume as temperature is decreased, or as the system ages. We clarify why such a behaviour is hard to explain within purely kinetic theories of glass formation, despite recent claims to the contrary.

We present a model equation of states for expanded metals, which contains a pressure term due to a screened-Coulomb potential with a screening parameter reflecting the Mott-Anderson metal-to-nonmetal transition. As anticipated almost 80 years ago by Zel'dovich and Landau, this term gives rise to a second coexistence line in the phase diagram, indicating a phase separation between a metallic and a nonmetallic liquid.

Although materials and processes are different from biological cells', brain mimicries led to tremendous achievements in massively parallel information processing via neuromorphic engineering. Inexistent in electronics, we describe how to emulate dendritic morphogenesis by electropolymerization in water, aiming in operando material modification for hardware learning. The systematic study of applied voltage-pulse parameters details on tuning independently morphological aspects of micrometric dendrites': as fractal number, branching degree, asymmetry, density or length. Time-lapse image processing of their growth shows the spatial features to be dynamically-dependent and expand distinctively before and after forming a conductive bridging of two electrochemically grown dendrites. Circuit-element analysis and electrochemical impedance spectroscopy confirms their morphological control to occur in temporal windows where the growth kinetics can be finely perturbed by the input signal frequency and duty cycle. By the emulation of one of the most preponderant mechanisms responsible for brain's long-term memory, its implementation in the vicinity of sensing arrays, neural probes or biochips shall greatly optimize computational costs and recognition performances required to classify high-dimensional patterns from complex aqueous environments.