Erik L. Johnson

Distributed clustering using collective principal component analysis

Abstract. This paper considers distributed clustering of high-dimensional heterogeneous data using a distributed principal component analysis (PCA) technique called the collective PCA. It presents the collective PCA technique, which can be used independent of the clustering application. It shows a way to integrate the Collective PCA with a given off-the-shelf clustering algorithm in order to develop a distributed clustering technique. It also presents experimental results using different test data sets including an application for web mining.

Collective, Hierarchical Clustering from Distributed, Heterogeneous Data

This paper presents the Collective Hierarchical Clustering (CHC) algorithm for analyzing distributed, heterogeneous data. This algorithm first generates local cluster models and then combines them to generate the global cluster model of the data. The proposed algorithm runs in O(|S|n2) time, with a O(|S|n) space requirement and O(n) communication requirement, where n is the number of elements in the data set and |S| is the number of data sites. This approach shows significant improvement over naive methods with O(n2) communication costs in the case that the entire distance matrix is transmitted and O(nm) communication costs to centralize the data, where m is the total number of features. A specific implementation based on the single link clustering and results comparing its performance with that of a centralized clustering algorithm are presented. An analysis of the algorithm complexity, in terms of overall computation time and communication requirements, is presented.

Distributed, Collaborative Data Analysis from Heterogeneous Sites Using a Scalable Evolutionary Technique

This paper documents an early effort to develop an experimental, collaborative data analysis technique for learning classifiers from a collection of heterogeneous datasets distributed over a network. The proposed technique makes use of a scalable evolutionary algorithm, called the GEMGA to classify datasets. This paper describes the developed technique and the results of the use of this technique through the application of this system for several domains, including distributed fault detection in an electrical power distribution network.

Analyzing the impact of social media on social movements: a computational study on Twitter and the occupy wall street movement

The extensive use of digital social media by social movement actors is an emerging trend that restructures the communication dynamics of social protest, and it is widely credited with contributing to the successful mobilizations of recent movements (e.g., Arab Spring, Occupy Wall Street). Yet, our understanding of both the roles played by social movements use of social media and the extent of its impact is largely derived from anecdotal evidence, news reports, and a thin body of scholarly research on Web-based technologies. In this research we explore several computational methods for measuring the impact of social media on a social movement. Inspired by methodologies originally developed for analyzing computer networks and other dynamic systems, these methods measure various static and dynamic aspects of social networks, and their relations to an underlying social movement. We demonstrated the feasibility and benefits of these measurement methods in the context of Twitter and the Occupying Wall Street movement (OWS). By analyzing tweets related to OWS, we demonstrated the link between the vitality of the movement and the volume of the related tweets over time. We show that there is a positive correlation between the dynamic of tweets and the short-term trend of OWS. The correlation makes it possible to forecast the short-term trend of a social movement using social media data. By ranking users based on the number of their OWS-related tweets and the durations of their tweeting, we are able to identify “buzz makers”. Using a strategy similar to the page-rank algorithm, we define the influence of a user by the number of re-tweets that his/her original tweets incite. By tracing where OWS-related tweets are generated, we measure the geographic diffusion of OWS. By analyzing the percentage of OWS tweets generated from different sources, we show that smart phones and applications such as tweet deck had been used extensively for tweeting in the OWS movement. This indicates the involvement of a younger and more technology-inclined generation in OWS.

Scheduling in optical WDM networks using hidden Markov chain-based traffic predictors

This paper presents the design and performance analysis of a predictor-based scheduling algorithm for optical wavelength division multiplexed (WDM) networks. A reservation-based multiple access control (MAC) protocol which schedules reservation requests from the network nodes on the multiple channels. We reduce the amount of time spent in computing the schedule by predicting traffic requests. The performance analysis based on discrete-event simulation, varying the parameters such as number of nodes and channels is presented.

Collective unsupervised data mining for heterogeneous wireless integrated network sensor arrays

Wireless integrated sensor networks involve information processing over large wired and wireless networks with limited bandwidth. Moreover, the computing capabilities of sensing devices are usually limited because of design restrictions, limited power supply, and other mission specific requirements. Analyzing data sets collected over such sensor networks usually requires downloading voluminous data sets to a central site. This fundamentally impairs the scalability and the overall response time of the application. In a mission critical applications, data analysis in such networks must deliver results within a certain time frame and often slower response completely defeats the purpose of analyzing sensor data. This paper presents a framework for collective data analysis from distributed heterogeneous data that calls for a fundamentally different perspective. This approach analyzes data in a distributed fashion without downloading everything to a central site. We examine several of the unsupervised Collective Data Mining algorithms for performing tasks associated with this extraction of useful information from sensor arrays. We further present a manner in which these algorithms may be incorporated into the knowledge extraction process from the sensor networks, and also propose an architecture geared towards the use of these algorithms.