Mário João Gonçalves Antunes

On using crowdsourcing and active learning to improve classification performance

Crowdsourcing is an emergent trend for general-purpose classification problem solving. Over the past decade, this notion has been embodied by enlisting a crowd of humans to help solve problems. There are a growing number of real-world problems that take advantage of this technique, such as Wikipedia, Linux or Amazon Mechanical Turk. In this paper, we evaluate its suitability for classification, namely if it can outperform state-of-the-art models by combining it with active learning techniques. We propose two approaches based on crowdsourcing and active learning and empirically evaluate the performance of a baseline Support Vector Machine when active learning examples are chosen and made available for classification to a crowd in a web-based scenario. The proposed crowdsourcing active learning approach was tested with Jester data set, a text humour classification benchmark, resulting in promising improvements over baseline results.

Defining Semantic Meta-hashtags for Twitter Classification

Given the wide spread of social networks, research efforts to retrieve information using tagging from social networks communications have increased. In particular, in Twitter social network, hashtags are widely used to define a shared context for events or topics. While this is a common practice often the hashtags freely introduced by the user become easily biased. In this paper, we propose to deal with this bias defining semantic meta-hashtags by clustering similar messages to improve the classification. First, we use the user-defined hashtags as the Twitter message class labels. Then, we apply the meta-hashtag approach to boost the performance of the message classification.

Concept Drift Awareness in Twitter Streams

Learning in non-stationary environments is not an easy task and requires a distinctive approach. The learning model must not only have the ability to continuously learn, but also the ability to acquired new concepts and forget the old ones. Additionally, given the significant importance that social networks gained as information networks, there is an ever-growing interest in the extraction of complex information used for trend detection, promoting services or market sensing. This dynamic nature tends to limit the performance of traditional static learning models and dynamic learning strategies must be put forward. In this paper we present a learning strategy to learn with drift in the occurrence of concepts in Twitter. We propose three different models: a time-window model, an ensemble-based model and an incremental model. Since little is known about the types of drift that can occur in Twitter, we simulate different types of drift by artificially time stamping real Twitter messages in order to evaluate and validate our strategy. Results are so far encouraging regarding learning in the presence of drift, along with classifying messages in Twitter streams.

A hybrid AIS-SVM ensemble approach for text classification

In this paper we propose and analyse methods for expanding state-of-the-art performance on text classification. We put forward an ensemble-based structure that includes Support Vector Machines (SVM) andArtificial Immune Systems (AIS).The underpinning idea is thatSVMlike approaches can be enhanced with AIS approaches which can capture dynamics in models. While having radically different genesis, and probably because of that, SVM and AIS can cooperate in a committee setting, using a heterogeneous ensemble to improve overall performance, including a confidence on each system classification as the differentiating factor. Results on the well-known Reuters-21578 benchmark are presented, showing promising classification performance gains, resulting in a classification that improves upon all baseline contributors of the ensemble committee.

TAT-NIDS: An Immune-Based Anomaly Detection Architecture for Network Intrusion Detection

One emergent, widely used metaphor and rich source of inspiration for computer security has been the vertebrate Immune System (IS). This is mainly due to its intrinsic nature of having to constantly protect the body against harm inflicted by external (non-self) harmful entities. The bridge between metaphor and the reality of new practical systems for anomaly detection is cemented by recent biological advancements and new proposed theories on the dynamics of immune cells by the field of theoretical immunology. In this paper we present a work in progress research on the deployment of an immune-inspired architecture, based on Grossman’s Tunable Activation Threshold (TAT) hypothesis, for temporal anomaly detection, where there is a strict temporal ordering on the data, such as network intrusion detection. We start by briefly describing the overall architecture. Then, we present some preliminary results obtained in a production network. Finally, we conclude by presenting the main lines of research we intend to pursue in the near future.

Automatic network configuration in virtualized environment using GNS3

Computer networking is a central topic in computer science courses curricula offered by higher education institutions. Network virtualization and simulation tools, like GNS3, allows students and practitioners to test real world networking configuration scenarios and to configure complex network scenarios by configuring virtualized equipments, such as routers and switches, through each ones virtual console. The configuration of advanced network topics in GNS3 requires that students have to apply basic and very repetitive IP configuration tasks in all network equipments. As the network topology grows, so does the amount of network equipments to be configured, which may lead to logical configuration errors. In this paper we propose an extension for GNS3 network virtualizer, to automatically generate a valid configuration of all the network equipments in a GNS3 scenario. Our implementation is able to automatically produce an initial IP and routing configuration of all the Cisco virtual equipments by using the GNS3 specification files. We tested this extension against a set of networked scenarios which proved the robustness, readiness and speedup of the overall configuration tasks. In a learning environment, this feature may save time for all networking practitioners, both beginners or advanced, who aim to configure and test network topologies, since it automatically produces a valid and operational configuration for all the equipments designed in a GNS3 environment.

Get your jokes right: ask the crowd

Jokes classification is an intrinsically subjective and complex task, mainly due to the difficulties related to cope with contextual constraints on classifying each joke. Nowadays people have less time to devote to search and enjoy humour and, as a consequence, people are usually interested on having a set of interesting filtered jokes that could be worth reading, that is with a high probability of make them laugh. In this paper we propose a crowdsourcing based collective intelligent mechanism to classify humour and to recommend the most interesting jokes for further reading. Crowdsourcing is becoming a model for problem solving, as it revolves around using groups of people to handle tasks traditionally associated with experts or machines. We put forward an active learning Support Vector Machine (SVM) approach that uses crowdsourcing to improve classification of user custom preferences. Experiments were carried out using the widely available Jester jokes dataset, with encouraging results.

Self Tolerance by Tuning T-Cell Activation: An Artificial Immune System for Anomaly Detection

The Artificial Immune Systems (AIS) constitute an emerging and very promising area of research that historically have been falling within two main theoretical immunological schools of thought: those based on Negative selection (NS) or those inspired on Danger theory (DT). Despite their inherent strengths and well known promising results, both deployed AIS have documented difficulties on dealing with gradual dynamic changes of self behavior through time.

The impact of longstanding messages in micro-blogging classification

Social networks are making part of the daily routine of millions of users. Twitter is among Facebook and Instagram one of the most used, and can be seen as a relevant source of information as users share not only daily status, but rapidly propagate news and events that occur worldwide. Considering the dynamic nature of social networks, and their potential in information spread, it is imperative to find learning strategies able to learn in these environments and cope with their dynamic nature. Time plays an important role by easily out-dating information, being crucial to understand how informative can past events be to current learning models and for how long it is relevant to store previously seen information, to avoid the computation burden associated with the amount of data produced. In this paper we study the impact of longstanding messages in micro-blogging classification by using different training time-window sizes in the learning process. Since there are few studies dealing with drift in Twitter and thus little is known about the types of drift that may occur, we simulate different types of drift in an artificial dataset to evaluate and validate our strategy. Results shed light on the relevance of previously seen examples according to different types of drift.

DOTS: Drift Oriented Tool System

Drift is a given in most machine learning applications. The idea that models must accommodate for changes, and thus be dynamic, is ubiquitous. Current challenges include temporal data streams, drift and non-stationary scenarios, often with text data, whether in social networks or in business systems. There are multiple drift patterns types: concepts that appear and disappear suddenly, recurrently, or even gradually or incrementally. Researchers strive to propose and test algorithms and techniques to deal with drift in text classification, but it is difficult to find adequate benchmarks in such dynamic environments.