Vincent Labatut

Comparative evaluation of community detection algorithms: a topological approach

Community detection is one of the most active fields in complex network analysis, due to its potential value in practical applications. Many works inspired by different paradigms are devoted to the development of algorithmic solutions allowing the network structure in such cohesive subgroups to be revealed. Comparative studies reported in the literature usually rely on a performance measure considering the community structure as a partition (Rand index, normalized mutual information, etc). However, this type of comparison neglects the topological properties of the communities. In this paper, we present a comprehensive comparative study of a representative set of community detection methods, in which we adopt both types of evaluation. Community-oriented topological measures are used to qualify the communities and evaluate their deviation from the reference structure. In order to mimic real-world systems, we use artificially generated realistic networks. It turns out there is no equivalence between the two approaches: a high performance does not necessarily correspond to correct topological properties, and vice versa. They can therefore be considered as complementary, and we recommend applying both of them in order to perform a complete and accurate assessment.

A Comparison of Community Detection Algorithms on Artificial Networks

Community detection has become a very important part in complex networks analysis. Authors traditionally test their algorithms on a few real or artificial networks. Testing on real networks is necessary, but also limited: the considered real networks are usually small, the actual underlying communities are generally not defined objectively, and it is not possible to control their properties. Generating artificial networks makes it possible to overcome these limitations. Until recently though, most works used variations of the classic Erdős-Renyi random model and consequently suffered from the same flaws, generating networks not realistic enough. In this work, we use Lancichinetti et al. model, which is able to generate networks with controlled power-law degree and community distributions, to test some community detection algorithms. We analyze the properties of the generated networks and use the normalized mutual information measure to assess the quality of the results and compare the considered algorithms.

Qualitative Comparison of Community Detection Algorithms

Community detection is a very active field in complex networks analysis, consisting in identifying groups of nodes more densely interconnected relatively to the rest of the network. The existing algorithms are usually tested and compared on real-world and artificial networks, their performance being assessed through some partition similarity measure. However, artificial networks realism can be questioned, and the appropriateness of those measures is not obvious. In this study, we take advantage of recent advances concerning the characterization of community structures to tackle these questions. We first generate networks thanks to the most realistic model available to date. Their analysis reveals they display only some of the properties observed in real-world community structures. We then apply five community detection algorithms on these networks and find out the performance assessed quantitatively does not necessarily agree with a qualitative analysis of the identified communities. It therefore seems both approaches should be applied to perform a relevant comparison of the algorithms.

A comparison of named entity recognition tools applied to biographical texts

Named entity recognition (NER) is a popular domain of natural language processing. For this reason, many tools exist to perform this task. Amongst other points, they differ in the processing method they rely upon, the entity types they can detect, the nature of the text they can handle, and their input/output formats. This makes it difficult for a user to select an appropriate NER tool for a specific situation. In this article, we try to answer this question in the context of biographic texts. For this matter, we first constitute a new corpus by annotating 247 Wikipedia articles. We then select 4 publicly available, well known and free for research NER tools for comparison: Stanford NER, Illinois NET, OpenCalais NER WS and Alias-i LingPipe. We apply them to our corpus, assess their performances and compare them. When considering overall performances, a clear hierarchy emerges: Stanford has the best results, followed by LingPipe, Illionois and OpenCalais. However, a more detailed evaluation performed relatively to entity types and article categories highlights the fact their performances are diversely influenced by those factors. This complementarity opens an interesting perspective regarding the combination of these individual tools in order to improve performance.

On Accuracy of Community Structure Discovery Algorithms

Community structure discovery in complex networks is a quite challenging problem spanning many applications in various disciplines such as biology, social network and physics. Emerging from various approaches numerous algorithms have been proposed to tackle this problem. Nevertheless little attention has been devoted to compare their efficiency on realistic simulated data. To better understand their relative performances, we evaluate systematically eleven algorithms covering the main approaches. The Normalized Mutual Information (NMI) measure is used to assess the quality of the discovered community structure from controlled artificial networks with realistic topological properties. Results show that along with the network size, the average proportion of intra-community to inter-community links is the most influential parameter on performances. Overall, Infomap is the leading algorithm, followed by Walktrap, SpinGlass and Louvain which also achieve good consistency.

The Effect of Network Realism on Community Detection Algorithms

Community detection consists in searching cohesive subgroups in complex networks. It has recently become one of the domain pivotal questions for scientists in many different fields where networks are used as modeling tools. Algorithms performing community detection are usually tested on real, but also on artificial networks, the former being costly and difficult to obtain. In this context, being able to generate networks with realistic properties is crucial for the reliability of the tests. Recently, Lancichinetti et al. [1] designed a method to produce realistic networks, with a community structure and power law distributed degrees and community sizes. However, other realistic properties such as degree correlation and transitivity are missing. In this work, we propose a modification of their approach, based on the preferential attachment model, in order to remedy this limitation. We analyze the properties of the generated networks and compare them to the original approach. We then apply different community detection algorithms and observe significant changes in their performances when compared to results on networks generated with the original approach.

Generalized Measures for the Evaluation of Community Detection Methods

Community detection can be considered as a variant of cluster analysis applied to complex networks. For this reason, all existing studies have been using tools derived from this field when evaluating community detection algorithms. However, those are not completely relevant in the context of network analysis, because they ignore an essential part of the available information: the network structure. Therefore, they can lead to incorrect interpretations. In this article, we review these measures, and illustrate this limitation. We propose a modification to solve this problem, and apply it to the three most widespread measures: purity, Rand index and normalised mutual information (NMI). We then perform an experimental evaluation on artificially generated networks with realistic community structure. We assess the relevance of the modified measures by comparison with their traditional counterparts, and also relatively to the topological properties of the community structures. On these data, the modified NMI turns out to provide the most relevant results.

Cerebral modeling and dynamic Bayesian networks

The understanding and the prediction of the clinical outcomes of focal or degenerative cerebral lesions, as well as the assessment of rehabilitation procedures, necessitate knowing the cerebral substratum of cognitive or sensorimotor functions. This is achieved by activation studies, where subjects are asked to perform a specific task while data of their brain functioning are obtained through functional neuroimaging techniques. Such studies, as well as animal experiments, have shown that sensorimotor or cognitive functions are the offspring of the activity of large-scale networks of anatomically connected cerebral regions. However, no one-to-one correspondence between activated networks adn functions can be found. Our research aims at understanding how the activation of large-scale networks derives from cerebral information processing mechanisms, which can only explain apparently conflicting activation data. Our work falls at the crossroads of neuroimaging interpretation techniques and computational neuroscience. Since knowledge in cognitive neuroscience is permanently evolving, our research aims more precisely at defining a new modeling formalism and at building a flexible simulator, allowing a quick implementation of the models, for a better interpretation of cerebral functional images. It also aims at providing plausible models, at eht level of large-scale networks, of cerebral information processing mechanisms in humans. In this paper, we propose a formalism, based on dynamic Bayesian networks (DBNs), that respects the following constraints: an oriented, networks architecture, whose nodes (the cerebral structures) can all be different, the implementation of causality--the activation of the structure is caused by upstream nodes activation--the explicit representation of different time scales (from 1 ms for the cerebral activity to many seconds for a PET scan image acquisition), the representation of cerebral information at the integrated level of neuronal populations, the imprecision of functional neuroimaging data, the nonlinearity and the uncertainty in cerebral mechanisms, and brains plasticity (learning, reorganization, modulation). One of the main problems, nonlinearity, has been tackled thanks to new extensions of the Kalman filter. The capabilities of the formalisms current version are illustrated by the modeling of a phoneme categorization process, explaining the different cerebral activations in normal and dyslexic subjects.

Classification of Complex Networks Based on Topological Properties

Study of countless real-world systems. They have been used in very different domains such as computer science, biology, sociology, management, etc. Authors have been trying to characterize them using various measures such as degree distribution, transitivity or average distance. Their goal is to detect certain properties such as the small-world or scale-free properties. Previous works have shown some of these properties are present in many different systems, while others are characteristic of certain types of systems only. However, each one of these studies generally focuses on a very small number of topological measures and networks. In this work, we aim at using a more systematic approach. We first constitute a dataset of 152 publicly available networks, spanning over 7 different domains. We then process 14 different topological measures to characterize them in the most possible complete way. Finally, we apply standard data mining tools to analyze these data. A cluster analysis reveals it is possible to obtain two significantly distinct clusters of networks, corresponding roughly to a bisection of the domains modeled by the networks. On these data, the most discriminant measures are density, modularity, average degree and transitivity, and at a lesser extent, closeness and edge betweenness centralities.

Detection and Interpretation of Communities in Complex Networks: Practical Methods and Application

Community detection is an important part of network analysis and has become a very popular field of research. This activity resulted in a profusion of community detection algorithms, all different in some not always clearly defined sense. This makes it very difficult to select an appropriate tool when facing the concrete task of having to identify and interpret groups of nodes, relatively to a system of interest. In this article, we tackle this problem in a very practical way, from the users point of view. We first review community detection algorithms and characterize them in terms of the nature of the communities they detect. We then focus on the methodological tools one can use to analyze the obtained community structure, both in terms of topological features and nodal attributes. To be as concrete as possible, we use a real-world social network to illustrate the application of the presented tools, and give examples of interpretation of their results from a Business Science perspective.