Tomoyuki Obuchi

Cross validation in LASSO and its acceleration

We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae.

Sparse approximation based on a random overcomplete basis

We discuss a strategy of sparse approximation that is based on the use of an overcomplete basis, and evaluate its performance when a random matrix is used as this basis. A small combination of basis vectors is chosen from a given overcomplete basis, according to a given compression rate, such that they compactly represent the target data with as small a distortion as possible. As a selection method, we study the

Spin and Chiral Orderings of the Antiferromagnetic XY Model on the Triangular Lattice and Their Critical Properties

\ell_0

Weight space structure and analysis using a finite replica number in the Ising perceptron

- and

Random versus maximum entropy models of neural population activity

\ell_1

Partition-function zeros of spherical spin glasses and their relevance to chaos

-based methods, which employ the exhaustive search and

Replica symmetry breaking, complexity and spin representation in the generalized random energy model

\ell_1

Accelerating cross-validation with total variation and its application to super-resolution imaging

-norm regularization techniques, respectively. The performance is assessed in terms of the trade-off relation between the representation distortion and the compression rate. First, we evaluate the performance analytically in the case that the methods are carried out ideally, using methods of statistical mechanics. Our result clarifies the fact that the

Learning probabilities from random observables in high dimensions: the maximum entropy distribution and others

\ell_0

Distribution of partition function zeros of the ±J model on the Bethe lattice

-based method greatly outperforms the