Shohei Shimizu

Letters: Estimation of linear non-Gaussian acyclic models for latent factors

Many methods have been proposed for discovery of causal relations among observed variables. But one often wants to discover causal relations among latent factors rather than observed variables. Some methods have been proposed to estimate linear acyclic models for latent factors that are measured by observed variables. However, most of the methods use data covariance structure alone for model identification, and this leads to a number of indistinguishable models. In this paper, we show that a linear acyclic model for latent factors is identifiable when the data are non-Gaussian.

Analyzing relationships among ARMA processes based on non-Gaussianity of external influences

The analysis of a relationship among variables in data generating systems is one of the important problems in machine learning. In this paper, we propose an approach for estimating a graphical representation of variables in data generating processes, based on the non-Gaussianity of external influences and an autoregressive moving-average (ARMA) model. The presented model consists of two parts, i.e., a classical structural-equation model for instantaneous effects and an ARMA model for lagged effects in processes, and is estimated through the analysis using the non-Gaussianity on the residual processes. As well as the recently proposed non-Gaussianity based method named LiNGAM analysis, the estimation by the proposed method has identifiability and consistency. We also address the relation of the estimated structure by our method to the Granger causality. Finally, we demonstrate analyses on the data containing both of the instantaneous causality and the Granger (temporal) causality by using our proposed method where the datasets for the demonstration cover both artificial and real physical systems.

An experimental comparison of linear non-Gaussian causal discovery methods and their variants

Many multivariate Gaussianity-based techniques for identifying causal networks of observed variables have been proposed. These methods have several problems such that they cannot uniquely identify the causal networks without any prior knowledge. To alleviate this problem, a non-Gaussianity-based identification method LiNGAM was proposed. Though the LiNGAM potentially identifies a unique causal network without using any prior knowledge, it needs to properly examine independence assumptions of the causal network and search the correct causal network by using finite observed data points only. On another front, a kernel based independence measure that evaluates the independence more strictly was recently proposed. In addition, some advanced generic search algorithms including beam search have been extensively studied in the past. In this paper, we propose some variants of the LiNGAM method which introduce the kernel based method and the beam search enabling more accurate causal network identification. Furthermore, we experimentally characterize the LiNGAM and its variants in terms of accuracy and robustness of their identification.

Assessing statistical reliability of LiNGAM via multiscale bootstrap

Structural equation models have been widely used to study causal relationships between continuous variables. Recently, a non-Gaussian method called LiNGAM was proposed to discover such causal models and has been extended in various directions. An important problem with LiNGAM is that the results are affected by the random sampling of the data as with any statistical method. Thus, some analysis of the confidence levels should be conducted. A common method to evaluate a confidence level is a bootstrap method. However, a confidence level computed by ordinary bootstrap is known to be biased as a probabilityvalue (p-value) of hypothesis testing. In this paper, we propose a new procedure to apply an advanced bootstrap method called multiscale bootstrap to compute p-values of LiNGAMoutputs. The multiscale bootstrap method gives unbiased p-values with asymptotic much higher accuracy. Experiments on artificial data demonstrate the utility of our approach.

Discovery of exogenous variables in data with more variables than observations

Many statistical methods have been proposed to estimate causal models in classical situations with fewer variables than observations. However, modern datasets including gene expression data increase the needs of high-dimensional causal modeling in challenging situations with orders of magnitude more variables than observations. In this paper, we propose a method to find exogenous variables in a linear non-Gaussian causal model, which requires much smaller sample sizes than conventional methods and works even when orders of magnitude more variables than observations. Exogenous variables work as triggers that activate causal chains in the model, and their identification leads to more efficient experimental designs and better understanding of the causal mechanism. We present experiments with artificial data and real-world gene expression data to evaluate the method.

Use of prior knowledge in a non-Gaussian method for learning linear structural equation models

We discuss causal structure learning based on linear structural equation models. Conventional learning methods most often assume Gaussianity and create many indistinguishable models. Therefore, in many cases it is difficult to obtain much information on the structure. Recently, a non-Gaussian learning method called LiNGAM has been proposed to identify the model structure without using prior knowledge on the structure. However, more efficient learning can be achieved if some prior knowledge on a part of the structure is available. In this paper, we propose to use prior knowledge to improve the performance of a state-of-art non-Gaussian method. Experiments on artificial data show that the accuracy and computational time are significantly improved even if the amount of prior knowledge is not so large.