Shun Kataoka

Inverse Problem in Pairwise Markov Random Fields Using Loopy Belief Propagation

In this paper, using Chebyshev polynomials, we derive a new representation of loopy belief propagation for pairwise Markov random fields in terms of moments. Using the new representation, we propose a new method to solve a statistical inverse problem in pairwise Markov random fields within the framework of loopy belief propagation. Our method allows us to solve the statistical inverse problem without any iterative operations. We numerically verify our method for the statistical inverse problem in Q -state Potts models.

Statistical performance analysis by loopy belief propagation in Bayesian image modeling

The mathematical structures of loopy belief propagation are reviewed for Bayesian image modeling from the standpoint of statistical mechanical informatics. We propose some schemes for evaluating the statistical performance of probabilistic binary image restoration. The schemes are constructed by means of the LBP, which is known as the Bethe approximation in statistical mechanics. We show some results of numerical experiments obtained by using the LBP algorithm as well as the statistical performance analysis for the probabilistic image restorations.

Statistical analysis of loopy belief propagation in random fields.

Loopy belief propagation (LBP), which is equivalent to the Bethe approximation in statistical mechanics, is a message-passing-type inference method that is widely used to analyze systems based on Markov random fields (MRFs). In this paper, we propose a message-passing-type method to analytically evaluate the quenched average of LBP in random fields by using the replica cluster variation method. The proposed analytical method is applicable to general pairwise MRFs with random fields whose distributions differ from each other and can give the quenched averages of the Bethe free energies over random fields, which are consistent with numerical results. The order of its computational cost is equivalent to that of standard LBP. In the latter part of this paper, we describe the application of the proposed method to Bayesian image restoration, in which we observed that our theoretical results are in good agreement with the numerical results for natural images.

Statistical analysis of the expectation-maximization algorithm with loopy belief propagation in Bayesian image modeling

New statistical approaches to hyperparameter estimation by means of the expectation-maximization (EM) algorithm and loopy belief propagation (LBP) are given for Bayesian image modeling from the standpoint of statistical-mechanical informatics. In the present paper, we give a new scheme for computing the average of the trajectory in the EM algorithm with LBP with respect to all the possible degraded images generated by the assumed degradation process from a given original image. Moreover, we also give a new scheme for computing the average of the trajectory in the EM algorithm with LBP with respect to all the possible original images generated by the assumed prior probability distribution. Bayesian image models for binary image restorations are constructed as Ising models with random external fields and spatially uniform nearest-neighbor interactions on the square lattice. Our schemes for computing the above random averages for some statistical quantities in LBP are constructed by means of the Bethe approximation for random spin systems. They are reduced to the solution of simultaneous integral equations for distributions of messages in LBP. We show the results of some numerical experiments in LBP as well as the statistical analysis of the trajectory in the EM algorithm for binary image restoration by LBP.

Statistical Analysis of Gaussian Image Inpainting Problems

Bayesian image processing is one of the activity research fields in computer science. Some of the approaches can often be reduced to inference problems on Markov random fields (MRFs). Image inpainting is one such problem. Image inpainting is used to infer intensities in missing regions from an observed image in which the intensities of some pixels are missing. In the inference scheme of inpainting, each observed image is divided into missing regions and nonmissing regions, and it is assumed that it is known which region each pixel belongs to in advance. A deterministic algorithm for the image inpainting problem based on Gaussian MRFs has been proposed. The proposed algorithm returns a unique output image for each observed image. By considering variations among observed images, we can define the statistical performances of the deterministic algorithm. The calculation of statistical performances corresponds to taking the average over all the possible observed images. Statistical performance analysis in probabilistic inference systems is one of the topics of statistical–mechanical informatics, and there are many successful results regarding statistical performance analysis in various research fields of computer sciences. Nishimori and Wong calculated the statistical performance in binary image restorations using MRFs defined on a complete graph. Statistical performance analysis in image restorations using Gaussian MRFs has also been calculated accurately by using the multi-dimensional Gauss integral formulas. Kataoka et al. have proposed approximate methods for evaluating the statistical performances and applied them for binary image restorations using MRFs on square grid graphs. In the present paper, we evaluate statistical performances of image inpaintings by Gaussian MRFs. Our evaluation method is an extension of the idea to evaluate the approximate statistical performance based on the loopy belief propagation in ref. 8, and the calculation of statistical performance is reduced to the problem of solving simultaneous integral equations of distributions of estimated values. We consider an image composed of N pixels on a square grid graph and define a state, corresponding to light intensities, for each pixel as any real number in ð 1;þ1Þ. N pixels are divided into missing regions and non-missing regions. Suppose that x 1⁄4 fxig and y 1⁄4 fyjg are sets of pixels in missing regions and non-missing regions, respectively. Here, i is a label at each pixel belonging to missing regions, whereas j is associated to a pixel in non-missing regions. In this paper, we consider an image in which each pixel is randomly missing with probability p. Let c 1⁄4 fcig; ci 2 f0; 1g; i 1⁄4 1; . . . ; N be a set of state variables that indicate whether pixels are missing or not. ci 1⁄4 0 denotes a pixel i belonging to non-missing regions and ci 1⁄4 1 denotes a pixel i belonging to missing regions. We assume that the posterior probability density function of x, under both c and y are given, is denoted by Pðx j c; yÞ 1⁄4 exp1⁄2 Hðx j c; yÞ =Z, where the energy function Hðx j c; yÞ is defined as Hðx j c; yÞ 1⁄4 1 2 X ði;jÞ2E 2 cicjðxi xjÞ

Community Detection Algorithm Combining Stochastic Block Model and Attribute Data Clustering

We propose a new algorithm to detect the community structure in a network that utilizes both the network structure and vertex attribute data. Suppose we have the network structure together with the vertex attribute data, that is, the information assigned to each vertex associated with the community to which it belongs. The problem addressed this paper is the detection of the community structure from the information of both the network structure and the vertex attribute data. Our approach is based on the Bayesian approach that models the posterior probability distribution of the community labels. The detection of the community structure in our method is achieved by using belief propagation and an EM algorithm. We numerically verified the performance of our method using computer-generated networks and real-world networks.

Bayesian Image Segmentations by Potts Prior and Loopy Belief Propagation

This paper presents a Bayesian image segmentation model based on Potts prior and loopy belief propagation. The proposed Bayesian model involves several terms, including the pairwise interactions of Potts models, and the average vectors and covariant matrices of Gauss distributions in color image modeling. These terms are often referred to as hyperparameters in statistical machine learning theory. In order to determine these hyperparameters, we propose a new scheme for hyperparameter estimation based on conditional maximization of entropy in the Potts prior. The algorithm is given based on loopy belief propagation. In addition, we compare our conditional maximum entropy framework with the conventional maximum likelihood framework, and also clarify how the first order phase transitions in LBPs for Potts models influence our hyperparameter estimation procedures.

Traffic data reconstruction based on Markov random field modeling

We consider the traffic data reconstruction problem. Suppose we have the traffic data of an entire city that are incomplete because some road data are unobserved. The problem is to reconstruct the unobserved parts of the data. In this paper, we propose a new method to reconstruct incomplete traffic data collected from various sensors. Our approach is based on Markov random field modeling of road traffic. The reconstruction is achieved by using a mean-field method and a machine learning method. We numerically verify the performance of our method using realistic simulated traffic data for the real road network of Sendai, Japan.

Inverse Renormalization Group Transformation in Bayesian Image Segmentations

A new Bayesian image segmentation algorithm is proposed by combining a loopy belief propagation with an inverse real space renormalization group transformation to reduce the computational time. In results of our experiment, we observe that the proposed method can reduce the computational time to less than one-tenth of that taken by conventional Bayesian approaches.

Momentum-Space Renormalization Group Transformation in Bayesian Image Modeling by Gaussian Graphical Model

A new Bayesian modeling method is proposed by combining the maximization of the marginal likelihood with a momentum-space renormalization group transformation for Gaussian graphical models. Moreover, we present a scheme for computint the statistical averages of hyperparameters and mean square errors in our proposed method based on a momentumspace renormalization transformation.