Yanning Shen

Nonlinear structural equation models for network topology inference

Linear structural equation models (SEMs) have been widely adopted for inference of causal interactions in complex networks. Recent examples include unveiling topologies of hidden causal networks over which processes such as spreading diseases, or rumors propagation. However, these approaches are limited because they assume linear dependence among observable variables. The present paper advocates a more general nonlinear structural equation model based on polynomial expansions, which compensates for possible nonlinear dependencies between network nodes. To this end, a group-sparsity regularized estimator is put forth to leverage the inherent edge sparsity that is present in most real-world networks. A novel computationally-efficient proximal gradient algorithm is developed to estimate the polynomial SEM coefficients, and hence infer the edge structure. Preliminary tests on simulated data demonstrate the effectiveness of the novel approach.

Inferring directed network topologies via tensor factorization

Directed networks are pervasive both in nature and engineered systems, often underlying the complex behavior observed in biological systems, microblogs and social interactions over the web, as well as global financial markets. Since their explicit structures are often unobservable, in order to facilitate network analytics, one generally resorts to approaches capitalizing on measurable nodal processes to infer the unknown topology. Prominent among these are structural equation models (SEMs), capable of incorporating exogenous inputs to resolve inherent directional ambiguities. However, this assumes full knowledge of exogenous inputs, which may not be readily available in some practical settings. The present paper advocates a novel SEM-based topology inference approach that entails a PARAFAC decomposition of a three-way tensor, constructed from the observed nodal data. It turns out that second-order statistics of exogenous variables suffice to identify the hidden topology. Leveraging the uniqueness properties inherent to high-order tensor factorizations, it is shown that topology identification is possible under reasonably mild conditions. Tests on simulated data corroborate the effectiveness of the novel tensor-based approach.

Topology inference of directed graphs using nonlinear structural vector autoregressive models

Linear structural vector autoregressive models constitute a generalization of structural equation models (SEMs) and vector autoregressive (VAR) models, two popular approaches for topology inference of directed graphs. Although simple and tractable, linear SVARMs seldom capture nonlinearities that are inherent to complex systems, such as the human brain. To this end, the present paper advocates kernel-based nonlinear SVARMs, and develops an efficient sparsity-promoting least-squares estimator to learn the hidden topology. Numerical tests on real electrocorticographic (ECoG) data from an Epilepsy study corroborate the efficacy of the novel approach.

Online Categorical Subspace Learning for Sketching Big Data with Misses

With the scale of data growing every day, reducing the dimensionality (a.k.a. sketching) of high-dimensional data has emerged as a task of paramount importance. Relevant issues to address in this context include the sheer volume of data that may consist of categorical observations, the typically streaming format of acquisition, and the possibly missing entries. To cope with these challenges, this paper develops a novel categorical subspace learning approach to unravel the latent structure for three prominent categorical (bilinear) models, namely, Probit, Tobit, and Logit. The deterministic Probit and Tobit models treat data as quantized values of an analog-valued process lying in a low-dimensional subspace, while the probabilistic Logit model relies on low dimensionality of the data log-likelihood ratios. Leveraging the low intrinsic dimensionality of the sought models, a rank regularized maximum-likelihood estimator is devised, which is then solved recursively via alternating majorization-minimization to sketch high-dimensional categorical data “on the fly.” The resultant lightweight first-order algorithms entail highly parallelizable tasks per iteration. In addition, the quantization thresholds are also learned jointly with the subspace to enhance the predictive power of the sought models. Performance of the subspace iterates is analyzed for both infinite and finite data streams, where for the former asymptotic convergence to the stationary point set of the batch estimator is established, while for the latter sublinear regret bounds are derived for the empirical cost. Simulated tests with both synthetic and real-world datasets corroborate the merits of the novel schemes for real-time movie recommendation and chess-game classification.

Kernel-Based Structural Equation Models for Topology Identification of Directed Networks

Structural equation models (SEMs) have been widely adopted for inference of causal interactions in complex networks. Recent examples include unveiling topologies of hidden causal networks over which processes, such as spreading diseases, or rumors propagate. The appeal of SEMs in these settings stems from their simplicity and tractability, since they typically assume linear dependencies among observable variables. Acknowledging the limitations inherent to adopting linear models, the present paper put forth nonlinear SEMs, which account for (possible) nonlinear dependencies among network nodes. The advocated approach leverages kernels as a powerful encompassing framework for nonlinear modeling, and an efficient estimator with affordable tradeoffs is put forth. Interestingly, pursuit of the novel kernel-based approach yields a convex regularized estimator that promotes edge sparsity, a property exhibited by most real world networks, and the resulting optimization problem is amenable to proximal-splitting optimization methods. To this end, solvers with complementary merits are developed by leveraging the alternating direction method of multipliers, and proximal gradient iterations. Experiments conducted on simulated data demonstrate that the novel approach outperforms linear SEMs with respect to edge detection errors. Furthermore, tests on a real gene expression dataset unveil interesting new edges that were not revealed by linear SEMs, which could shed more light on regulatory behavior of human genes.

Tensor Decompositions for Identifying Directed Graph Topologies and Tracking Dynamic Networks

Directed networks are pervasive both in nature and engineered systems, often underlying the complex behavior observed in biological systems, microblogs and social interactions over the web, as well as global financial markets. Since their structures are often unobservable, in order to facilitate network analytics, one generally resorts to approaches capitalizing on measurable nodal processes to infer the unknown topology. Structural equation models (SEMs) are capable of incorporating exogenous inputs to resolve inherent directional ambiguities. However, conventional SEMs assume full knowledge of exogenous inputs, which may not be readily available in some practical settings. This paper advocates a novel SEM-based topology inference approach that entails factorization of a three-way tensor, constructed from the observed nodal data, using the well-known parallel factor (PARAFAC) decomposition. It turns out that second-order piecewise stationary statistics of exogenous variables suffice to identify the hidden topology. Capitalizing on the uniqueness properties inherent to high-order tensor factorizations, it is shown that topology identification is possible under reasonably mild conditions. In addition, to facilitate real-time operation and inference of time-varying networks, an adaptive (PARAFAC) tensor decomposition scheme that tracks the topology-revealing tensor factors is developed. Extensive tests on simulated and real stock quote data demonstrate the merits of the novel tensor-based approach.

Online sketching of big categorical data with absent features

With the scale of data growing every day, reducing the dimensionality (a.k.a. sketching) of high-dimensional vectors has emerged as a task of increasing importance. Relevant issues to address in this context include the sheer volume of data vectors that may consist of categorical (meaning finite-alphabet) features, the typically streaming format of data acquisition, and the possibly absent features. To cope with these challenges, the present paper brings forth a novel rank-regularized maximum likelihood approach that models categorical data as quantized values of analog-amplitude features with low intrinsic dimensionality. This model along with recent online rank regularization advances are leveraged to sketch high-dimensional categorical data `on the fly. Simulated tests with synthetic as well as real-world datasets corroborate the merits of the novel scheme relative to state-of-the-art alternatives.

Topology inference of multilayer networks

Linear structural equation models (SEMs) have been very successful in identifying the topology of complex graphs, such as those representing tactical, social and brain networks. The rising popularity of multilayer networks, presents the need for tools that are tailored to leverage the layered structure of the underlying network. To this end, a multilayer SEM is put forth, to infer causal relations between nodes belonging to multilayer networks. An efficient algorithm based on the alternating direction method of multipliers (ADMM) is developed, and preliminary tests on synthetic as well as real data demonstrate the effectiveness of the proposed approach.

Tracking dynamic piecewise-constant network topologies via adaptive tensor factorization

This paper deals with tracking dynamic piecewise-constant network topologies that underpin complex systems including online social networks, neural pathways in the brain, and the world-wide web. Leveraging a structural equation model (SEM) in which only second-order statistics of exogenous inputs are known, the topology inference problem is recast using three-way tensors constructed from observed nodal data. To facilitate real-time operation, an adaptive parallel factor (PARAFAC) tensor decomposition is advocated to track the topology-revealing tensor factors. Preliminary tests on simulated data corroborate the effectiveness of the novel tensor-based approach.

Online dictionary learning from large-scale binary data

Compressive sensing (CS) has been shown useful for reducing dimensionality, by exploiting signal sparsity inherent to specific domain representations of data. Traditional CS approaches represent the signal as a sparse linear combination of basis vectors from a prescribed dictionary. However, it is often impractical to presume accurate knowledge of the basis, which motivates data-driven dictionary learning. Moreover, in large-scale settings one may only afford to acquire quantized measurements, which may arrive sequentially in a streaming fashion. The present paper jointly learns the sparse signal representation and the unknown dictionary when only binary streaming measurements with possible misses are available. To this end, a novel efficient online estimator with closed-form sequential updates is put forth to recover the sparse representation, while refining the dictionary `on the fly. Numerical tests on simulated and real data corroborate the efficacy of the novel approach.