Nima Reyhani

Consistency and asymptotic normality of FastICA and bootstrap FastICA

Independent component analysis (ICA) is possibly the most widespread approach to solve the blind source separation problem. Many different algorithms have been proposed, together with several highly successful applications. There is also an extensive body of work on the theoretical foundations and limits of the ICA methodology. One practical concern about the use of ICA with real world data is the robustness of its estimates. Slight variations in the estimates, may stem from the inherent stochastic nature of the algorithms used or some deviations from the theoretical assumptions. To overcome this problem, different approaches have been proposed, most of which are based on the use of multiple runs of ICA algorithms with bootstrap. Here we show the consistency and asymptotic normality of FastICA and bootstrap FastICA, based on empirical process theory, including Z-estimators and Hoeffdings inequality. These results give theoretical grounds for the robust use of FastICA, in a multiple run, bootstrap and randomly initialized manner. In this framework, it is also possible to assess the convergence of the algorithm through a normality test.

Multiple Kernel Learning by Conditional Entropy Minimization

Kernel methods have been successfully used in many practical machine learning problems. Choosing a suitable kernel is left to the practitioner. A common way to an automatic selection of optimal kernels is to learn a linear combination of element kernels. In this paper, a novel framework of multiple kernel learning is proposed based on conditional entropy minimization criterion. For the proposed framework, three multiple kernel learning algorithms are derived. The algorithms are experimentally shown to be comparable to or outperform kernel Fisher discriminant analysis and other multiple kernel learning algorithms on benchmark data sets.

Multiple kernel learning with gaussianity measures

Kernel methods are known to be effective for nonlinear multivariate analysis. One of the main issues in the practical use of kernel methods is the selection of kernel. There have been a lot of studies on kernel selection and kernel learning. Multiple kernel learning (MKL) is one of the promising kernel optimization approaches. Kernel methods are applied to various classifiers including Fisher discriminant analysis (FDA). FDA gives the Bayes optimal classification axis if the data distribution of each class in the feature space is a gaussian with a shared covariance structure. Based on this fact, an MKL framework based on the notion of gaussianity is proposed. As a concrete implementation, an empirical characteristic function is adopted to measure gaussianity in the feature space associated with a convex combination of kernel functions, and two MKL algorithms are derived. From experimental results on some data sets, we show that the proposed kernel learning followed by FDA offers strong classification power.

Cancer-Predicting Gene Expression Changes in Colonic Mucosa of Western Diet Fed Mlh1+/- Mice

Colorectal cancer (CRC) is the second most common cause of cancer-related deaths in the Western world and interactions between genetic and environmental factors, including diet, are suggested to play a critical role in its etiology. We conducted a long-term feeding experiment in the mouse to address gene expression and methylation changes arising in histologically normal colonic mucosa as putative cancer-predisposing events available for early detection. The expression of 94 growth-regulatory genes previously linked to human CRC was studied at two time points (5 weeks and 12 months of age) in the heterozygote Mlh1 +/- mice, an animal model for human Lynch syndrome (LS), and wild type Mlh1 +/+ littermates, fed by either Western-style (WD) or AIN-93G control diet. In mice fed with WD, proximal colon mucosa, the predominant site of cancer formation in LS, exhibited a significant expression decrease in tumor suppressor genes, Dkk1, Hoxd1, Slc5a8, and Socs1, the latter two only in the Mlh1 +/- mice. Reduced mRNA expression was accompanied by increased promoter methylation of the respective genes. The strongest expression decrease (7.3 fold) together with a significant increase in its promoter methylation was seen in Dkk1, an antagonist of the canonical Wnt signaling pathway. Furthermore, the inactivation of Dkk1 seems to predispose to neoplasias in the proximal colon. This and the fact that Mlh1 which showed only modest methylation was still expressed in both Mlh1 +/- and Mlh1 +/+ mice indicate that the expression decreases and the inactivation of Dkk1 in particular is a prominent early marker for colon oncogenesis.

Inherited cancer predisposition sensitizes colonic mucosa to address Western diet effects and putative cancer-predisposing changes on mouse proteome☆ , ☆☆

Human epidemiological evidence and previous studies on mice have shown that Western-style diet (WD) may predispose gut mucosa to colorectal cancer (CRC). The mechanisms that mediate the effects of diet on tumorigenesis are largely unknown. To address putative cancer-predisposing events available for early detection, we quantitatively analyzed the proteome of histologically normal colon of a wild-type (Mlh1(+/+)) and an Mlh1(+/-) mouse after a long-term feeding experiment with WD and AIN-93G control diet. The Mlh1(+/-) mouse carries susceptibility to colon cancer analogous to a human CRC syndrome (Lynch syndrome). Remarkably, WD induced expression changes reflecting metabolic disturbances especially in the cancer-predisposed colon, while similar changes were not significant in the wild-type proteome. Overall, the detected changes constitute a complex interaction network of proteins involved in ATP synthesis coupled proton transport, oxidoreduction coenzyme and nicotinamide nucleotide metabolic processes, important in cell protection against reactive oxygen species toxicity. Of these proteins, selenium binding protein 1 and galectin-4, which directly interact with MutL homolog 1, are underlined in neoplastic processes, suggesting that sensitivity to WD is increased by an Mlh1 mutation. The significance of WD on CRC risk is highlighted by the fact that five out of six mice with neoplasias were fed with WD.

An iterative algorithm for singular value decomposition on noisy incomplete matrices

In this paper, we propose a simple iterative algorithm, called iSVD, for estimating the singular value decomposition (SVD) of a noisy incomplete given matrix. The iSVD relies on first order optimization over orthogonal manifolds and automatically estimates the rank of the SVD. The main goal here is to estimate the singular vectors through optimization in the right space, which is the space of the orthogonal matrix manifolds. The rank estimation is based on the ratio between estimated large singular values and the sum of all singular values. We empirically evaluate the iSVD on synthetic matrices and image reconstruction tasks. The evaluation shows that the iSVD is comparable to the recently introduced methods for matrix completion such as singular value thresholding (SVT) and fixed-point iteration with approximate SVD (FPCA).

Speaker recognition using multiple kernel learning based on conditional entropy minimization

We applied a multiple kernel learning (MKL) method based on information-theoretic optimization to speaker recognition. Most of the kernel methods applied to speaker recognition systems require a suitable kernel function and its parameters to be determined for a given data set. In contrast, MKL eliminates the need for strict determination of the kernel function and parameters by using a convex combination of element kernels. In the present paper, we describe an MKL algorithm based on conditional entropy minimization (MCEM). We experimentally verified the effectiveness of MCEM for speaker classification; this method reduced the speaker error rate as compared to conventional methods.

Multiple spectral kernel learning and a gaussian complexity computation

Multiple kernel learning (MKL) partially solves the kernel selection problem in support vector machines and similar classifiers by minimizing the empirical risk over a subset of the linear combination of given kernel matrices. For large sample sets, the size of the kernel matrices becomes a numerical issue. In many cases, the kernel matrix is of low-efficient rank. However, the low-rank property is not efficiently utilized in MKL algorithms. Here, we suggest multiple spectral kernel learning that efficiently uses the low-rank property by finding a kernel matrix from a set of Gram matrices of a few eigenvectors from all given kernel matrices, called a spectral kernel set. We provide a new bound for the gaussian complexity of the proposed kernel set, which depends on both the geometry of the kernel set and the number of Gram matrices. This characterization of the complexity implies that in an MKL setting, adding more kernels may not monotonically increase the complexity, while previous bounds show otherwise.

Supervised covariance selection for linear discriminant analysis

Linear discriminant analysis relies on sample covariance matrix, which is a major estimation issue in many high dimensional statistical problems. Sample covariance matrix estimation has been studied recently and a number of solutions are proposed for such problem. Naïve Bayes approach assumes that the covariates are independent in high dimensional settings and showed that this assumption theoretically results in high classification accuracy. Here, we study the performance of other covariance estimators when the sparseness is not assumed to be huge, which comes at some computational cost compared to naïve Bayes. Our study shows that in some cases, we might gain by taking a covariance matrix with controlled sparseness. Then, we cast the covariance selection problem into the framework of empirical risk minimization, and propose the supervised covariance learning which uses the labels information in covariance matrix selection. The empirical results show that the use of controlled sparseness and labels information improves the classification accuracy compared to the naïve Bayes.

Abstract B18: Western diet-fed mice with inherited cancer predisposition reveal early epigenetic changes and protein markers for colon oncogenesis

Colorectal cancer (CRC) is the second most common cause of cancer-related deaths in the Western world. Interactions between genetic and environmental factors, such as diet, are suggested to play a critical role in its etiology. However, the mechanisms that mediate the effects of diet on oncogenesis are largely unknown. We conducted a long-term feeding experiment in the mouse to address epigenetic changes arising in normal colonic mucosa and their functional consequences as putative cancer-predisposing events available for early detection. The quantitative analysis of proteomes and the expression of 94 growth-regulatory genes previously linked to human CRC and aberrant hypermethylation were studied at two time points (5 weeks and 12 months of age) in the heterozygote Mlh1+/- mice analogous to human Lynch syndrome, and wild type Mlh1+/+ littermates, fed with Western-style (WD) or AIN-93G control diet. In the proteome study the most significant expression changes were found in the Mlh1+/- WD mice compared to mice fed with AIN-93G control diet and the 19 identified proteins mainly point to abnormities in energy metabolism and protein unfolding. We further found that in mice carrying Mlh1 mutation and/or fed with WD, histologically normal proximal colonic mucosa exhibited a significant expression decrease in tumor suppressor genes, Dkk1, Slc5a8, Hoxd1, and Socs1. Especially in Dkk1, a secreted antagonist of the Wnt/β-catenin pathway the reduced expression was associated with its promoter hypermethylation. The fact that changes in Dkk1 seem to predispose to neoplasia in the proximal colon and that 5 out of 6 mice with neoplastic colonic lesions were fed with WD suggests that the found expression changes are early markers for oncogenesis. Citation Format: Denis Dermadi Bebek, Satu Valo, Marjaana Pussila, Nima Reyhani, Laura Sarantaus, Minna Nystrom. Western diet-fed mice with inherited cancer predisposition reveal early epigenetic changes and protein markers for colon oncogenesis. [abstract]. In: Proceedings of the AACR Special Conference on Chromatin and Epigenetics in Cancer; Jun 19-22, 2013; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2013;73(13 Suppl):Abstract nr B18.