Pedro A. Forero

Consensus-based distributed linear support vector machines

This paper develops algorithms to train linear support vector machines (SVMs) when training data are distributed across different nodes and their communication to a centralized node is prohibited due to, for example, communication overhead or privacy reasons. To accomplish this goal, the centralized linear SVM problem is cast as the solution of coupled decentralized convex optimization subproblems with consensus constraints on the parameters defining the classifier. Using the method of multipliers, distributed training algorithms are derived that do not exchange elements from the training set among nodes. The communications overhead of the novel approach is fixed and fully determined by the topology of the network instead of being determined by the size of the training sets as it is the case for existing incremental approaches. An online algorithm where data arrive sequentially to the nodes is also developed. Simulated tests illustrate the performance of the algorithms.

Distributed Clustering Using Wireless Sensor Networks

Clustering spatially distributed data is well motivated and especially challenging when communication to a central processing unit is discouraged, e.g., due to power constraints. Distributed clustering schemes are developed in this paper for both deterministic and probabilistic approaches to unsupervised learning. The centralized problem is solved in a distributed fashion by recasting it to a set of smaller local clustering problems with consensus constraints on the cluster parameters. The resulting iterative schemes do not exchange local data among nodes, and rely only on single-hop communications. Performance of the novel algorithms is illustrated with simulated tests on synthetic and real sensor data. Surprisingly, these tests reveal that the distributed algorithms can exhibit improved robustness to initialization than their centralized counterparts.

Consensus-based distributed expectation-maximization algorithm for density estimation and classification using wireless sensor networks

The present paper develops a decentralized expectation-maximization (EM) algorithm to estimate the parameters of a mixture density model for use in distributed learning tasks performed with data collected at spatially deployed wireless sensors. The E-step in the novel iterative scheme relies on local information available to individual sensors, while during the M-step sensors exchange information only with their one- hop neighbors to reach consensus and eventually percolate the global information needed to estimate the wanted parameters across the wireless sensor network (WSN). Analysis and simulations demonstrate that the resultant consensus-based distributed EM (CB-DEM) algorithm matches well the resource- limited characteristics of WSNs and compares favorably with existing alternatives because it has wider applicability and remains resilient to inter-sensor communication noise.

Distributed feature-based modulation classification using wireless sensor networks

Automatic modulation classification (AMC) is a critical prerequisite for demodulation of communication signals in tactical scenarios. Depending on the number of unknown parameters involved, the complexity of AMC can be prohibitive. Existing maximum-likelihood and feature-based approaches rely on centralized processing. The present paper develops AMC algorithms using spatially distributed sensors, each acquiring relevant features of the received signal. Individual sensors may be unable to extract all relevant features to reach a reliable classification decision. However, the cooperative in-network approach developed enables high classification rates at reduced-overhead, even when features are noisy and/or missing. Simulated tests illustrate the performance of the novel distributed AMC scheme.

Sparsity-Exploiting Robust Multidimensional Scaling

Multidimensional scaling (MDS) seeks an embedding of N objects in a p <; N dimensional space such that inter-vector distances approximate pairwise object dissimilarities. Despite their popularity, MDS algorithms are sensitive to outliers, yielding grossly erroneous embeddings even if few outliers contaminate the available dissimilarities. This work introduces robust MDS approaches exploiting the degree of sparsity in the outliers present. Links with compressive sampling lead to robust MDS solvers capable of coping with unstructured and structured outliers. The novel algorithms rely on a majorization-minimization approach to minimize a regularized stress function, whereby iterative MDS solvers involving Lasso and sparse group-Lasso operators are obtained. The resulting schemes identify outliers and obtain the desired embedding at computational cost comparable to that of their nonrobust MDS alternatives. The robust structured MDS algorithm considers outliers introduced by a sparse set of objects. In this case, two types of sparsity are exploited: i) sparsity of outliers in the dissimilarities; and ii) sparsity of the objects introducing outliers. Numerical tests on synthetic and real datasets illustrate the merits of the proposed algorithms.

Robust Clustering Using Outlier-Sparsity Regularization

Notwithstanding the popularity of conventional clustering algorithms such as K-means and probabilistic clustering, their clustering results are sensitive to the presence of outliers in the data. Even a few outliers can compromise the ability of these algorithms to identify meaningful hidden structures rendering their outcome unreliable. This paper develops robust clustering algorithms that not only aim to cluster the data, but also to identify the outliers. The novel approaches rely on the infrequent presence of outliers in the data, which translates to sparsity in a judiciously chosen domain. Leveraging sparsity in the outlier domain, outlier-aware robust K-means and probabilistic clustering approaches are proposed. Their novelty lies on identifying outliers while effecting sparsity in the outlier domain through carefully chosen regularization. A block coordinate descent approach is developed to obtain iterative algorithms with convergence guarantees and small excess computational complexity with respect to their non-robust counterparts. Kernelized versions of the robust clustering algorithms are also developed to efficiently handle high-dimensional data, identify nonlinearly separable clusters, or even cluster objects that are not represented by vectors. Numerical tests on both synthetic and real datasets validate the performance and applicability of the novel algorithms.

Shallow-water sparsity-cognizant source-location mapping

Using passive sonar for underwater acoustic source localization in a shallow-water environment is challenging due to the complexities of underwater acoustic propagation. Matched-field processing (MFP) exploits both measured and model-predicted acoustic pressures to localize acoustic sources. However, the ambiguity surface obtained through MFP contains artifacts that limit its ability to reveal the location of the acoustic sources. This work introduces a robust scheme for shallow-water source localization that exploits the inherent sparse structure of the localization problem and the use of a model characterizing the acoustic propagation environment. To this end, the underwater acoustic source-localization problem is cast as a sparsity-inducing stochastic optimization problem that is robust to model mismatch. The resulting source-location map (SLM) yields reduced ambiguities and improved resolution, even at low signal-to-noise ratios, when compared to those obtained via classical MFP approaches. An iterative solver based on block-coordinate descent is developed whose computational complexity per iteration is linear with respect to the number of locations considered for the SLM. Numerical tests illustrate the performance of the algorithm.

Prediction of Partially Observed Dynamical Processes Over Networks via Dictionary Learning

Prediction of dynamical processes evolving over network graphs is a basic task encountered in various areas of science and engineering. The prediction challenge is exacerbated when only partial network observations are available, that is when only measurements from a subset of nodes are available. To tackle this challenge, the present work introduces a joint topology- and data-driven approach for network-wide prediction able to handle partially observed network data. First, the known network structure and historical data are leveraged to design a dictionary for representing the network process. The novel approach draws from semi-supervised learning to enable learning the dictionary with only partial network observations. Once the dictionary is learned, network-wide prediction becomes possible via a regularized least-squares estimate which exploits the parsimony encapsulated in the design of the dictionary. Second, an online network-wide prediction algorithm is developed to jointly extrapolate the process over the network and update the dictionary accordingly. This algorithm is able to handle large training datasets at a fixed computational cost. More important, the online algorithm takes into account the temporal correlation of the underlying process, and thereby improves prediction accuracy. The performance of the novel algorithms is illustrated for prediction of real Internet traffic. There, the proposed approaches are shown to outperform competitive alternatives.

Rollout Algorithms for Data Storage- and Energy-Aware Data Retrieval Using Autonomous Underwater Vehicles

Increasingly effective underwater networks will be required to meet the growing demand for undersea data. The impending exploitation of non-acoustic underwater communication modes and the proliferation of autonomous underwater vehicles (AUVs) will enable the development of underwater networks able to use multiple modes of wireless communications and AUVs to transport data. In this paradigm, planning the routes for AUVs to collect data from underwater sensors becomes critical due to the dynamic nature of the undersea environment and the data collection process. This work proposes a dynamic path planning framework that enables judicious decisions on which network nodes the AUVs should visit next, based on the most recent network-status information. Routing decisions are aware of the AUVs own data-storage and energy constraints. Motivated by the intractability of optimal AUV routing, we propose a rollout algorithm as an enabler for dynamic AUV routing. Numerical tests illustrate the performance of the proposed algorithm.

Joint link learning and cognitive radio sensing

Novel cooperative spectrum sensing algorithms for cognitive radios (CRs) are developed, which can blindly learn the channel gains between CRs and licensed primary users (PUs), while jointly detecting active PU transmitters at each time instant. A dictionary learning approach is taken to decompose the received signal energy samples per CR into linear combinations of channel gains and PU transmit-powers, up to scaling ambiguity. In addition to a batch baseline algorithm, an efficient online implementation that can track slow variation of channel gains is developed, as well as a distributed alternative, which requires only local message passing among neighbors in CR networks. Numerical tests verify the proposed design.