Dennis M. Wilkinson

E-Mail as Spectroscopy: Automated Discovery of Community Structure within Organizations

We describe a method for the automatic identification of communities of practice from e-mail logs within an organization. We use a betweenness centrality algorithm that can rapidly find communities within a graph representing information flows. We apply this algorithm to an initial e-mail corpus of nearly 1 million messages collected over a 2-month span, and show that the method is effective at identifying true communities, both formal and informal, within these scale-free graphs. This approach also enables the identification of leadership roles within the communities. These studies are complemented by a qualitative evaluation of the results in the field.

Rhythms of Social Interaction: Messaging Within a Massive Online Network

College students spend a significant amount of time using online social net- work services for messaging, sharing information, and keeping in touch with one another (e.g. [3, 10]). As these services represent a plentiful source of electronic data, they provide an opportunity to study dynamic patterns of social interactions quickly and exhaustively. In this paper, we study the social net- work service Facebook, which began in early 2004 in select universities, but grew quickly to encompass a very large number of universities. Studies have shown that, as of 2006, Facebook use is nearly ubiquitous among U. S. college students with over 90% active participation among undergraduates [5, 16].

Large-Scale Parallel Collaborative Filtering for the Netflix Prize

Many recommendation systems suggest items to users by utilizing the techniques of collaborative filtering(CF) based on historical records of items that the users have viewed, purchased, or rated. Two major problems that most CF approaches have to contend with are scalability and sparseness of the user profiles. To tackle these issues, in this paper, we describe a CF algorithm alternating-least-squares with weighted-?-regularization(ALS-WR), which is implemented on a parallel Matlab platform. We show empirically that the performance of ALS-WR (in terms of root mean squared error(RMSE)) monotonically improves with both the number of features and the number of ALS iterations. We applied the ALS-WR algorithm on a large-scale CF problem, the Netflix Challenge, with 1000 hidden features and obtained a RMSE score of 0.8985, which is one of the best results based on a pure method. In addition, combining with the parallel version of other known methods, we achieved a performance improvement of 5.91% over Netflixs own CineMatch recommendation system. Our method is simple and scales well to very large datasets.

Email as spectroscopy: automated discovery of community structure within organizations

We describe a method for the automatic identification of communities of practice from email logs within an organization. We use a betweenness centrality algorithm that can rapidly find communities within a graph representing information flows. We apply this algorithm to an email corpus of nearly one million messages collected over a two-month span, and show that the method is effective at identifying true communities, both formal and informal, within these scale-free graphs. This approach also enables the identification of leadership roles within the communities. These studies are complemented by a qualitative evaluation of the results in the field.

Cooperation and quality in wikipedia

The rise of the Internet has enabled collaboration and cooperation on anunprecedentedly large scale. The online encyclopedia Wikipedia, which presently comprises 7.2 million articles created by 7.04 million distinct editors, provides a consummate example. We examined all 50 million edits made tothe 1.5 million English-language Wikipedia articles and found that the high-quality articles are distinguished by a marked increase in number of edits, number of editors, and intensity of cooperative behavior, as compared to other articles of similar visibility and age. This is significant because in other domains, fruitful cooperation has proven to be difficult to sustain as the size of the collaboration increases. Furthermore, in spite of the vagaries of human behavior, we show that Wikipedia articles accrete edits according to a simple stochastic mechanism in which edits beget edits. Topics of high interest or relevance are thus naturally brought to the forefront of quality.

Strong regularities in online peer production

Online peer production systems have enabled people to coactively create, share, classify, and rate content on an unprecedented scale. This paper describes strong macroscopic regularities in how people contribute to peer production systems, and shows how these regularities arise from simple dynamical rules. First, it is demonstrated that the probability a person stops contributing varies inversely with the number of contributions he has made. This rule leads to a power law distribution for the number of contributions per person in which a small number of very active users make most of the contributions. The rule also implies that the power law exponent is proportional to the effort required to contribute, as justified by the data. Second, the level of activity per topic is shown to follow a lognormal distribution generated by a stochastic reinforcement mechanism. A small number of very popular topics thus accumulate the vast majority of contributions. These trends are demonstrated to hold across hundreds of millions of contributions to four disparate peer production systems of differing scope, interface style, and purpose.

Hierarchical Structure and Search in Complex Organizations

Organizations engage in search whenever they perform nonroutine tasks, such as the definition and validation of a new strategy, the acquisition of new capabilities, or new product development. Previous work on search and organizational hierarchy has discovered that a hierarchy with a central decision maker at the top can speed up problem solving, but possibly at the cost of solution quality compared with results of a decentralized search. Our study uses a formal model and simulations to explore the effect of an organizational hierarchy on solution stability, solution quality, and search speed. Three insights arise on how a hierarchy can improve organizational search: (1) assigning a lead function that “anchors” a solution speeds up problem solving; (2) local solution choice should be delegated to the lowest level; and (3) structure matters little at the middle management level, but it matters at the front line; front-line groups should be kept small. These results highlight the importance for every organization of adapting its hierarchical structure to its search requirements.

Feedback Loops of Attention in Peer Production

A significant percentage of online content is now published and consumed via the mechanism of crowdsourcing. While any user can contribute to these forums, a disproportionately large percentage of the content is submitted by very activeand devoted users, whose continuing participation is key to thesites’ success. As we show, people’s propensity to keep participating increases the more they contribute, suggesting motivating factors which increase over time. This paper demonstrates that submitters who stop receiving attention tend to stop contributing, while prolific contributors attract an ever increasing number offollowers and their attention in a feedback loop. We demonstrate that this mechanism leads to the observed power law in the number of contributions per user and support our assertions by an analysis of hundreds of millions of contributions to top content sharing websites Digg.com and Youtube.com.

A literature based method for identifying gene-disease connections

We present a statistical method that can swiftly identify, from the literature, sets of genes known to be associated with given diseases. It offers a comprehensive way to treat alias symbols, a statistical method for computing the relevance of the gene to the query, and a novel way to disambiguate gene symbols from other abbreviations. The method is illustrated by finding genes related to breast cancer.

Performance Variability and Project Dynamics

We present a dynamical model of complex cooperative projects such as large engineering design or software development efforts, comprised of concurrent and interrelated tasks. The model contains a stochastic component to account for temporal fluctuations both in task performance and in the interactions between related tasks. We show that as the system size increases, so does the average completion time. Also, for fixed system size, the dynamics of individual project realizations can exhibit large deviations from the average when fluctuations increase past a threshold, causing long delays in completion times. These effects are in agreement with empirical observation. We also show that the negative effects of both large groups and long delays caused by fluctuations may be mitigated by arranging projects in a hierarchical or modular structure. Our model is applicable to any arrangement of interdependent tasks, providing an analytical prediction for the average completion time as well as a numerical threshold for the fluctuation strength beyond which long delays are likely. In conjunction with previous modeling techniques, it thus provides managers with a predictive tool to be used in the design of a projects architecture.