Haroon Raja

Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data

This paper studies the problem of data-adaptive representations for big, distributed data. It is assumed that a number of geographically-distributed, interconnected sites have massive local data and they are interested in collaboratively learning a low-dimensional geometric structure underlying these data. In contrast with previous works on subspace-based data representations, this paper focuses on the geometric structure of a union of subspaces (UoS). In this regard, it proposes a distributed algorithm-termed cloud K-SVD-for collaborative learning of a UoS structure underlying distributed data of interest. The goal of cloud K-SVD is to learn a common overcomplete dictionary at each individual site such that every sample in the distributed data can be represented through a small number of atoms of the learned dictionary. Cloud K-SVD accomplishes this goal without requiring exchange of individual samples between sites. This makes it suitable for applications where sharing of raw data is discouraged due to either privacy concerns or large volumes of data. This paper also provides an analysis of cloud K-SVD that gives insights into its properties as well as deviations of the dictionaries learned at individual sites from a centralized solution in terms of different measures of local/global data and topology of interconnections. Finally, the paper numerically illustrates the efficacy of cloud K-SVD on real and synthetic distributed data.

Cloud K-SVD: Computing data-adaptive representations in the cloud

This paper studies the problem of data-adaptive representations for big, distributed data. It is assumed that a number of geographically-distributed, interconnected sites have massive local data and they are interested in collaboratively learning a low-dimensional geometric structure underlying these data. In contrast to some of the previous works on subspace representations, this paper focuses on the geometric structure of a union of subspaces (UoS). Specifically, it proposes a distributed algorithm, termed as cloud K-SVD, for learning a UoS structure underlying distributed data of interest. Cloud K-SVD accomplishes the goal of collaborative data-adaptive representations without requiring communication of individual data samples between different sites. The paper also provides a partial analysis of cloud K-SVD that gives insights into its convergence properties and deviations from a centralized solution in terms of properties of local data and topology of interconnections. Finally, it numerically illustrates the efficacy of cloud K-SVD.

Performance analysis of WiMAX best effort and ertPS service classes for video transmission

To support different types of data like http, real-time audio and video, VoIP, FTP, there are various classes in WiMax system. In this work, we try to analyze the performance when multimedia contents are transmitted over WiMax network. Due to stringent delay requirement of real-time multimedia data, a separate class is allocated for it. i.e. rtPS. Thus our objective is to find out that how much we gain advantage by transmitting multimedia over this separate class? This requires a thorough analysis while considering all the scenarios. Our contribution in this paper is to build an initial framework for answering the above stated questions. The Network Simulator (ns-2) which is a popular tool for the simulation of computer networks has been used to simulate the results. Standard-compliant implementations have been used to authenticate the results.

Parametric dictionary learning for TWRI using distributed particle swarm optimization

This paper considers a distributed network of through-the-wall radars for accurate indoor scene reconstruction in the presence of multipath propagation. A sparsity based method is proposed for eliminating ghost targets under imperfect knowledge of interior wall locations. Instead of aggregating and processing the observations at a central fusion station, joint scene reconstruction and estimation of interior wall locations is carried out in a distributed manner across the network. More specifically, an alternating minimization approach is utilized to solve the associated non-convex optimization problem, wherein the sparse scene is reconstructed using the recently proposed modified distributed orthogonal matching pursuit algorithm while the wall location estimates are obtained with a novel distributed particle swarm optimization algorithm (D-PSO) proposed in this paper. Existing literature on averaging consensus is leveraged to derive the D-PSO algorithm. The efficacy of proposed approach is demonstrated using numerical simulation.

A convergence analysis of distributed dictionary learning based on the K-SVD algorithm

This paper provides a convergence analysis of a recent distributed algorithm, termed cloud K-SVD, that solves the problem of data-adaptive representations for big, distributed data. It is assumed that a number of geographically-distributed, interconnected sites have massive local data and they are collaboratively learning a sparsifying dictionary underlying these data using cloud K-SVD. This paper provides a rigorous analysis of cloud K-SVD that gives insights into its properties as well as deviations of the dictionaries learned at individual sites from a centralized solution in terms of different measures of local/global data and topology of the interconnections.

Dictionary learning based nonlinear classifier training from distributed data

This paper addresses the problem of collaborative training of nonlinear classifiers using big, distributed training data. The supervised learning strategy considered in this paper corresponds to data-driven joint learning of a nonlinear transformation that maps the (training) data to a higher-dimensional feature space and a ridge regression based linear classifier in the feature space. The key aspect of this paper, which distinguishes it from related prior work, is that it assumes: (i) the training data are distributed across a number of interconnected sites, and (ii) sizes of the local training data as well as privacy concerns prohibit exchange of individual training samples between sites. The main contribution of this paper is formulation of an algorithm, termed cloud D-KSVD, that reliably, efficiently and collaboratively learns both the nonlinear map and the linear classifier under these constraints. In order to demonstrate the effectiveness of cloud D-KSVD, a number of numerical experiments on the MNIST dataset are also reported in the paper.

Throughput enhancement by cross-layer header compression in WLANs

A major limiting factor in increasing the throughput of wireless networks has been the bottleneck of prohibitive signaling overhead. A number of header compression schemes have been proposed to solve this particular problem. These, however, come with their set of limitations and suffer from a loss of practicality in certain cases. Our contribution in this paper is two-fold; firstly, we propose a practical framework for cross-layer header compression which improves vastly on existing data rates (by more than 25% of raw throughput at high SNRs). This is achieved by applying independent compression algorithms to both MAC and PHY data units, thus ensuring better throughput. Secondly, we analyze the performance of proposed scheme in slow fading as well as fast fading environments to demonstrate robustness.

Rate-Distortion optimized transcoder selection for multimedia transmission in heterogeneous networks

In this paper we propose a solution for selection of appropriate transcoding nodes in a network operating in ad-hoc mode. The heterogeneity present in todays networked devices necessitates different quality of video for different end users. One possible solution for this heterogeneity is to transcode the video stream as per user demand. In this work, we define significant parameters to facilitate decision on selection of transcoding nodes within a wireless access network. We formulate the problem as a rate-distortion optimization to achieve conflicting objectives of high quality and minimum time of delivery to an end user. Unlike past works which have focused on transcoding to develop efficient distributed transcoders, our aim is to come up with methods for placement of these parallel transcoding nodes in a heterogeneous network, keeping in view the constraints of timely delivery of video and minimal distortion.

Recent developments in distributed dictionary learning

Most of the research on dictionary learning has focused on developing algorithms under the assumption that data is available at a centralized location. But often the data is not available at a centralized location due to practical constraints like data aggregation costs, privacy concerns, etc. Using centralized dictionary learning algorithms may not be the optimal choice in such settings. This motivates the design of dictionary learning algorithms that consider distributed nature of data as one of the problem variables. Just like centralized settings, distributed dictionary learning problem can be posed in more than one way depending on the problem setup. Most notable distinguishing features are the online versus batch nature of data and the representative versus discriminative nature of the dictionaries. In this paper, several distributed dictionary learning algorithms that are designed to tackle different problem setups are reviewed. One of these algorithms is cloud K-SVD, which solves the dictionary learning problem for batch data in distributed settings. One distinguishing feature of cloud K-SVD is that it has been shown to converge to its centralized counterpart, namely, the K-SVD solution. On the other hand, no such guarantees are provided for other distributed dictionary learning algorithms. Convergence of cloud K-SVD to the centralized K-SVD solution means problems that are solvable by K-SVD in centralized settings can now be solved in distributed settings with similar performance. Finally, cloud K-SVD is used as an example to show the advantages that are attainable by deploying distributed dictionary algorithms for real world distributed datasets.

Learning overcomplete representations from distributed data: a brief review

Most of the research on dictionary learning has focused on developing algorithms under the assumption that data is available at a centralized location. But often the data is not available at a centralized location due to practical constraints like data aggregation costs, privacy concerns, etc. Using centralized dictionary learning algorithms may not be the optimal choice in such settings. This motivates the design of dictionary learning algorithms that consider distributed nature of data as one of the problem variables. Just like centralized settings, distributed dictionary learning problem can be posed in more than one way depending on the problem setup. Most notable distinguishing features are the online versus batch nature of data and the representative versus discriminative nature of the dictionaries. In this paper, several distributed dictionary learning algorithms that are designed to tackle different problem setups are reviewed. One of these algorithms is cloud K-SVD, which solves the dictionary learning problem for batch data in distributed settings. One distinguishing feature of cloud K-SVD is that it has been shown to converge to its centralized counterpart, namely, the K-SVD solution. On the other hand, no such guarantees are provided for other distributed dictionary learning algorithms. Convergence of cloud K-SVD to the centralized K-SVD solution means problems that are solvable by K-SVD in centralized settings can now be solved in distributed settings with similar performance. Finally, cloud K-SVD is used as an example to show the advantages that are attainable by deploying distributed dictionary algorithms for real world distributed datasets.