Kazushi Mimura

Synapse efficiency diverges due to synaptic pruning following overgrowth

In the development of the brain, it is known that synapses are pruned following overgrowth. This pruning following overgrowth seems to be a universal phenomenon that occurs in almost all areas-visual cortex, motor area, association area, and so on. It has been shown numerically that the synapse efficiency is increased by systematic deletion. We discuss the synapse efficiency to evaluate the effect of pruning following overgrowth, and analytically show that the synapse efficiency diverges as O(|ln c|) at the limit where connecting rate c is extremely small. Under a fixed synapse number criterion, the optimal connecting rate, which maximizes memory performance, exists.

Parallel dynamics of disordered Ising spin systems on finitely connected directed random graphs with arbitrary degree distributions

We study the stochastic parallel dynamics of Ising spin systems defined on finitely connected directed random graphs with arbitrary degree distributions, using generating functional analysis. For fully asymmetric graphs the dynamics of the system can be completely solved, due to the asymptotic absence of loops. For arbitrary graph symmetry, we solve the dynamics exactly for the first few time steps, and we construct approximate stationary solutions.

Generating functional analysis of CDMA detection dynamics

We investigate the detection dynamics of the parallel interference canceller (PIC) for code-division multiple-access (CDMA) multiuser detection, applied to a randomly spread, fully synchronous base-band uncoded CDMA channel model with additive white Gaussian noise (AWGN) under perfect power control in the large-system limit. It is known that the predictions of the density evolution (DE) can fairly explain the detection dynamics only in the case where the detection dynamics converge. At transients, though, the predictions of DE systematically deviate from computer simulation results. Furthermore, when the detection dynamics fail to converge, the deviation of the predictions of DE from the results of numerical experiments becomes large. As an alternative, generating functional analysis (GFA) can take into account the effect of the Onsager reaction term exactly and does not need the Gaussian assumption of the local field. We present GFA to evaluate the detection dynamics of PIC for CDMA multiuser detection. The predictions of GFA exhibit good consistency with the computer simulation result for any condition, even if the dynamics fail to converge.

The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles

An exact solution of the transient dynamics of an associative memory model storing an infinite number of limit cycles with l finite steps is shown by means of the path-integral analysis. Assuming the Maxwell construction ansatz, we have succeeded in deriving the stationary state equations of the order parameters from the macroscopic recursive equations with respect to the finite-step sequence processing model which has retarded self-interactions. We have also derived the stationary state equations by means of the signal-to-noise analysis (SCSNA). The signal-to-noise analysis must assume that crosstalk noise of an input to spins obeys a Gaussian distribution. On the other hand, the path-integral method does not require such a Gaussian approximation of crosstalk noise. We have found that both the signal-to-noise analysis and the path-integral analysis give completely the same result with respect to the stationary state in the case where the dynamics is deterministic, when we assume the Maxwell construction ansatz. We have shown the dependence of the storage capacity (αc) on the number of patterns per one limit cycle (l). At l = 1, the storage capacity is αc = 0.138 as in the Hopfield model. The storage capacity monotonically increases with the number of steps, and converges to αc = 0.269 at l 10. The original properties of the finite-step sequence processing model appear as long as the number of steps of the limit cycle has order l = O(1).

Compressed sensing MRI using sparsity induced from adjacent slice similarity

We propose a fast magnetic resonance imaging (MRI) technique based on compressed sensing. The main idea is to use a combination of full and compressed sensing. Full sensing is conducted for every several slices (F-slice) while compressed sensing with high compression rate is applied to the rest of slices (C-slice). We can perfectly reconstruct F-slice images, which are used to roughly estimate the C-slices. Since the estimate is already of good quality, its difference from the original image is small and sparse. Therefore, the difference can be reconstructed precisely using the standard compressed sensing technique even with high compression rate. Simulation results show that the proposed method outperforms conventional methods with 3.16dB for arm images, 0.26dB for brain images in average for the C-slices with perfect reconstruction for the F-slices.

Statistical mechanics of lossy compression for non-monotonic multilayer perceptrons

A lossy data compression scheme for uniformly biased Boolean messages is investigated via statistical mechanics techniques. The present paper utilize tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions are non-monotonic, completing the study of the lossy compression scheme using perceptron-based decoder. The scheme performance at the infinite code length limit is analyzed using the replica method. Both committee and parity treelike networks are shown to saturate the Shannon bound.

The typical performance of irregular low-density generator-matrix codes for lossy compression

We evaluate typical performance of irregular low-density generator-matrix (LDGM) codes, which is defined by sparse matrices with arbitrary irregular bit degree distribution and arbitrary check degree distribution, for lossy compression. We apply the replica method under one-step replica symmetry breaking (1RSB) ansatz to this problem.

Statistical mechanics of lossy compression using multilayer perceptrons.

Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a treelike committee machine (committee tree) and treelike parity machine (parity tree) whose transfer functions are monotonic. For compression using a committee tree, a lower bound of achievable distortion becomes small as the number of hidden units K increases. However, it cannot reach the Shannon bound even where K-->infinity. For a compression using a parity tree with K> or =2 hidden units, the rate distortion function, which is known as the theoretical limit for compression, is derived where the code length becomes infinity.

Sparsely encoded associative memory: static synaptic noise and static threshold noise

In the present paper, an associative memory model with sparse coding is analyzed by means of the self-consistent signal-to-noise analysis (SCSNA). We discuss some effects of sparseness and the shape of the response function on memory capacity, considering a case using monotonic neurons. The memory capacity strongly depends on the shape of the response function, as well as sparseness. Moreover, a model with static synaptic noise and static noise in the threshold is discussed.

Generating Functional Analysis for Iterative CDMA Multiuser Detectors

We investigate the detection dynamics of a soft parallel interference canceler (soft-PIC), which includes a hard-PIC as a special case, for code-division multiple-access (CDMA) multiuser detection, applied to a randomly spread, fully synchronous base-band uncoded CDMA channel model with additive white Gaussian noise under perfect power control in the large-system limit. We analyze the detection dynamics of some iterative detectors, namely soft-PIC, the Onsager-reaction-canceling parallel interference canceler (ORC-PIC) and the belief-propagation-based detector (BP-based detector), by the generating functional analysis (GFA). The GFA allows us to study the asymptotic behavior of the dynamics in the infinitely large system without assuming the independence of messages. We study the detection dynamics and the stationary estimates of an iterative algorithm. We also show the decoupling principle in iterative multiuser detection algorithms in the large-system limit. For a generic iterative multiuser detection algorithm with binary input, it is shown that the multiuser channel is equivalent to a bank of independent single-user additive non-Gaussian channels, whose signal-to-noise ratio degrades due to both the multiple-access interference and the Onsager reaction, at each stage of the algorithm. If an algorithm cancels the Onsager reaction, the equivalent single-user channels coincide with an additive white Gaussian noise channel. We also discuss ORC-PIC and the BP-based detector.